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1 . Introduction 


f 


In  this  monograph give^a  mathematical  analysis  of 
spectral  methods  for  mixed  initial-boundary  value  problems. 

Spectral  methods  have  become  increasingly  popular  in  recent 
years,  especially  since  the  development  of  fast  transform 
methods  Setr.  10 T,  with  applications  in  numerical  weather 

prediction,  numerical  simulations  of  turbulent  flows,  and  other 
problems  where  high  accuracy  is  desired  for  complicated  solutions, 

We  do  not  discuss  the  sophisticated  applications  of  spectral 
methods  here;  a survey  of  some  applications  is  given  in  Sec.  15  . 
Instead,  we  concentrate  on  ^the  development  of  a mathematical  theory 
that  explains  why  spectral  methods  work  and  how  well  they  work. 
Before  presenting  the  theory,  we  begin  by  giving  some  simple 
examples  of  the  kinds  of  behavior  that  we  wish  to  explain. 

Spectral  methods  involve  representing  the  solution  to  a 
problem  as  a truncated  series  of  known  functions  of  the  inde- 
pendent variables.  We  shall  make  this  idea  precise  in  Sec.  2, 
but  we  can  illustrate  it  here  by  the  standard  separation  of 
variables  solution  to  the  mixed  initial-boundary  value  problem 
for  the  heat  equation. 

Example  1.1;  Fourier  sine  series  solution  of  the  heat  equation. 
Consider  the  mixed  initial-boundary  value  problem 


3u (x, t) 

3 1 

8 u(x,t) 

8x* 

(0  < 

X < IT,  t ^ 0) 

(1.1a) 

u (0, t)  = 

u(ir,t)  = 0 

(t  > 

0) 

(1.1b) 

u (x, 0)  - 

f (X) 

(0  < 

x < it)  . 

(1.1c) 

mmi 
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Thm  solution  to  (1.1)  is 

cx> 

u(x,t)  = l a (t)  sin  n x , (1.2) 

n*1 

an(t)  = fn  e n (n*l» 2, . . . , ) (1.3) 


where 


sin  nx  dx 


(n*l ,2, . . . ,) 


(1.4) 


are  the  coefficients  of  the  Fourier  sine  series  expansion  of 
f(x).  Recall  that  any  function  in  L2(0,tt)  has  a Fourier  sine 
series  that  converges  to  it  in  I^lOfir);  the  Fourier  sine  series 
of  any  piecewise  continuous  function  f(x)  which  has  bounded 
variation  on  (0,tt)  converges  to  j[f (x+)+f (x-) ] throughout 
(0, it)  (tee  Sec.  3). 


(1.2) 


A spectral  approximation  is  gotten  by  simply  truncating 
to 

N 


uN(x,t)  = l an(t)sin  nx 
n=l 


(1.3) 


and  replacing  (1.3)  by  the  evolution  equation 


dan 

■ar  = ' 


n2a 


n 


(n=l , . . . ,N)  . 


(1.6) 


with  the  initial  conditions  an(0)  = fn  (n=l,...,N)  . 

The  spectral  approximation  (1.5-6)  to  (1.1)  is  an  ex- 
ceedingly good  approximation  for  any  t > 0 as  N > ® 

In  fact,  the  error  u(x,t)  - u^(x,t)  goes  to  zero  more  rapidly 
than  e as  N -*•  <*>  for  any  t > 0 . in  contrast,  a finite 

difference  approximation  to  the  heat  equation  using  N grid  points 
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( 


< 


heat  equation. 

Not  all  spectral  methods  work  as  well  as  the  trivial  one 
just  outlined  in  Example  1.1.  Consider  for  example  the  solution 
to  the  problem 


9u 
3 1 


(0  < X < TT  , t>0) 


with  the  same  initial  and  boundary  conditions  as  before. 

The  Fourier  sine  coefficients  of  the  exact  solution  are  now 


an  ( t> 


f 


n 


t 


(l-e"n2t)e. 


(1.7) 


where  e = 0 if  n is  even  and  e_  — 1 if  n is  odd.  Spectral 
n ** 

approximations  are  now  given  by  (1.5)  with  (1.6)  replaced  by 


da 

n 


2 

n an 


(n=l,...,N)  , 


l 
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the  solution  of  which  is  (1.7)  for  n * 1,...,N.  Now  the 

truncation  error  u(x,t)  - uN(x,t)  no  longer  decays  exponentially 

as  N «*  ; the  error  is  of  order  N ^ as  N -*•  00  for  fixed 

x , 0 < x < s , and  t > 0 . In  other  words,  the  results 

to  be  anticipated  from  this  spectral  method  behave  asymptotically 

as  N ■*  “ in  the  same  way  as  those  obtained  by  a third-order 

finite-difference  scheme  [in  which  the  error  goes  to  zero  like 

Ax^  ■ (tt/N)^].  for  this  problem,  straightforward  solution  by 
finite  differences  may  be  more  efficient  and  accurate  than  solution 
by  Fourier  series. 

The  last  example  may  be  disturbing  but  even  more  serious 
difficulties  confront  the  unwary  user  of  spectral  methods,  as 
the  next  example  should  make  amply  clear. 

Example  1.3:  Fourier  sine  series  solution  of  the  one- 
dimensional wave  equation. 

Consider  the  mixed  initial-boundary  value  problem  for  the 
one-dimensional  wave  equation 

= x + t (0  < x < n , t > 0)  (1.8a) 

u ( 0,  t)  ■ 0 (t  > 0)  (1.8b) 

u (x, 0)  =0  (0  < x < n)  (1.8c) 

The  exact  solution  to  this  well  posed  problem  is  u(x,t)  = xt . 

This  solution  can  also  be  found  by  Fourier  sine  series  expansion 
of  u(x,t).  To  do  this,  we  substitute  (1.2)  into  (1.8)  and  re- 
expand all  terms  in  sine  series.  The  Fourier  expansion  of  9u/9x  is 


= V b (t)ain  nx 
8x  nil  n 


where  integration  by  parts  gives 

b (t)  - 3-  J a—  sin  nx  dx,  - - -p  / u cos  nx  dx 

n o dx  0 

oo  n 

« - y a It)  I Sin  mx  cos  nx  dx, 

It  n»  'n 


(1.  9) 


(1.10) 


7 i %lt»  • 


m«l 
m+n  odd 


n -m 


, n+1 


Also  the  Fourier  sine  coefficients  of  x are  2/n(-l)  and 

the  Fourier  sine  coefficients  of  t are  (4t/irn)en,  where  en  = 0 
if  n is  even  and  e^  =*  1 if  n is  odd.  Equating  coefficients 
of  sin  nx  in  (1.8a)  we  obtain 


dan 

W~ 


- I 

n m=l 
m+n  odd 


nm 

2 2 ~m 

n -m 


a . ! (_i)n  + -i  te  (n*l,2,...), 
— n it  n n 


(1.11) 


The  Fourier  sine  coefficients  of  the  exact  solution 
u(x,t)  * xt  are 


an<t)  * 


* n 


(n  = 1,2,...) 


It  is  easy  to  verify  by  direct  substitution  that  these  coefficients 
satisfy  (1.11)  exactly;  in  particular,  the  sum  in  (1. 11)  converges 
for  all  t. 
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Now  suppose  we  employ  a spectral  method  based  on  Fourier 
sine  series  to  solve  this  problem.  We  seek  a solution  to  (1.8)  in 
the  form  of  the  truncated  sine  series  (1.4) . if  the  exact  co- 
efficients «n(t)  are  used  in  (1.4)  then  u(x,t)  - uN(x,t)  -♦  0 
as  N -♦  <*>  ; for  each  fixed  x , 0 < x < tt  , and  t > 0 the 
error  is  of  order  1/N  as  N -*■  °®  (see  Sec.  3). 

However,  it  is  not  reasonable  to  assume  that  the  expansion 
coefficients  an(t)  are  known  exactly  in  this  case  because  of 
the  complicated  couplings  between  various  n in  the  system 
(1.111  It  is  more  reasonable  to  determine  them  by  numerical 
solution  of  an  approximation  to  (1.11).  Galerkin  approximation 
(see  Sec.  2)  gives  the  truncated  system  of  equations 


dan 

dt~ 


N 

1 I 

If  L . 


m=l 
m+n  odd 


nm 

zrj 

n -m 


am  ' 


1 (-1)"  + ten  (n-1 m 


(1.12) 


The  truncation  of  the  infinite  system  (1.11)  to  the  finite 
system  (1.12)  is  a standard  way  to  approximate  infinite  coupled 
systems.  Unfortunately,  it  need  not  work.  In  Figs.  1.1-1. 2 
we  show  plots  of  the  approximations  uN(x,t)  at  t * 5 given 
by  (1.4)  for  N * 50,75.  These  plots  are  obtained  by  numerical 
solution  of  (1.12)  with  an(0)  = 0;  the  time  steps  used  in  the 
numerical  solution  of  (1.12)  are  so  small  that  time  differencing 
errors  are  negligible.  It  is  apparent  that  the  approximate  solu- 


for  N«50  at  t«*5.  This  solution  is  obtainted  by  numerical  Integration  of 
(1.12).  Time  differencing  errors  are  negligible.  The  exact  solution 
xt  at  t “5  is  also  shown. 


Fig.  1.2.  A plot  of  the  Galerkin  approximation  u^Cx.t)  to  (1.8) 
for  N*75  at  t=5.  This  solution  is  obtained  by  numerical 


tions  with  N finite  do  not  converge  to  the  exact  solution  as  N 
increases  1 The  divergence  of  this  spectral  method  will  be  ex- 
plained in  Sec.  6. 

Not  all  spectral  methods  give  such  poor  results.  A properly 
formulated  and  implemented  spectral  method  gives  results  of 
striking  accuracy  with  efficient  use  of  computer  resources. 

The  choice  of  an  appropriate  spectral  method  is  go/erned  by 
two  main  considerations: 

(i)  Accuracy.  in  order  to  be  useful  a 
spectral  method  should  be  designed  to  give  results 
of  greater  accuracy  than  can  be  obtained  by 
more  conventional  difference  methods  using  similar 
spatial  resolution  or  degrees  of  freedom.  The  choice 
of  appropriate  spectral  representation  depends  on  the 
kind  of  boundary  conditions  involved  in  the  problem. 

(ii)  Efficiency.  In  order  to  be  useful  the  spec- 
tral method  should  be  as  efficient  as  difference 

methods  with  comparable  numbers  of  degrees  of 
freedom.  For  similar  work,  spectral  methods 
should  produce  more  accurate  results  than 
conventional  methods. 

In  Sec.  15,  we  present  a catalog  of  different  spectral  methods 
and  indicate  the  kinds  of  problems  to  which  they  can  be  most  use- 
fully applied. 

Many  examples  of  efficient  and  accurate  spectral  methods  will 


be  given  later. 


2.  Spectral  Methods 

The  problems  to  be  studied  here  are  mixed  initial-boundary 
value  problems  of  the  form 

?u%t I = L(x>t)U(x,t)  + f(x,t)  (x  e D , t > 0)  (2.1) 

a t 

B(x)u(x,t)  =0  (x  e 3D,  t > 0)  (2.2) 

u(x,  0)  = g(x)  (x  e D)  (2.3) 

where  D is  a spatial  domain  with  boundary  3D  , L(x,t)  is  a 
linear  (spatial)  differential  operator  and  B(x)  is  a linear 
(time  independent)  boundary  operator.  Here  we  write  (2.1-3) 
for  a single  dependent  variable  u and  a single  space  coordinate 
x with  the  understanding  that  much  of  the  following  analy- 
sis generalizes  to  systems  of  equations  in  higher  space  di- 
mensions. Also,  attention  is  restricted  to  problems  with 
homogeneous  boundary  conditions  because  the  solution  to  any 
problem  involving  inhomogeneous  boundary  conditions  is  the  sum  of 
an  arbitrary  function  having  the  imposed  boundary  values  and 
a solution  to  a problem  of  the  form  (2.1-3).  The  extension  to 
nonlinear  problems  will  be  indicated  at  the  end  of  this  section. 

Before  discussing  spectral  methods  for  solution  of  (2.1-3)  let 
us  set  up  the  mathematical  framework  for  our  later  analysis. 

It  is  assumed  that,  for  each  t , u(x,t)  is  an  element 

of  a Hilbert  space  H with  inner  product  ( , ) and  norm 
||  . ||  . For  each  t > 0 , the  solution1  u(t)  belongs  to 

the  subspace  8 of  H consisting  of  all  functions  u e H 

1 We  will  often  denote  u(x,t)  by  u(t)  when  discussing  u as 
a function  of  t. 
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t. 


a 


! 


( 


( 


■j’syyn:.. 


satisfying  Bu  = 0 on  3D  . We  do  not  require  that  u (x, 0) -g (x) e B 

but  only  that  u(x,0)  c hf  . The  operator  I,  is  typically  an 

unbounded  dif f erentiul  operator  whose  domain  is  dense  in, 

but  smaller  than,  . For  example,  if 

L = 3/3x  and  LjtOjl),  the  domain  of  L can  be 

chosen  as  the  dense  set  of  all  absolutely  continuous  functions  on 

0 < x _<  1 

If  the  problem  (2.1-3)  is  well  posed,  the  evolution  operator 
is  a bounded  linear  operator  from  H to  8 . Boundedness  implies 

that  the  domain  of  the  evolution  operator  can  be  extended  in 

a standard  v;ay  from  the  domain  of  L to  the  whole  space  H 
(Richtmyer  & Morton,  1967,  p.  34).  For  notational  convenience 

we  shall  assume  henceforth  that  L is  time  independent  so  that 
the  evolution  operator  is  exp(Lt).  in  this  case  the  formal  so- 
lution of  (2.1-3)  is 

t 

u ( t ) = eLtu (0)  + j cL(t_s) f (s)ds  (2.4) 

This  formal  solution  is  justified  under  the  conditions 
that  f(t)  , Lf(t)  , and  L f(t)  exist  and  arc  continuous 
functions  of  t in  the  norm  | | • | | for  all  t - 0 (see 
Richtmyer  & Morton,  1967) . 

The  semi-discrete  approximations  to  (2.1)  to  be  studied  here 
arc  of  the  form 


( 
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r 


■i'rjr- 


T ^ 


a u ( x , t ) 

N 

at 


LN  UN(X,t)  + fN(x,t) 


(2.5) 


where,  for  each  t , uN(x,t)  belongs  to  an  N-dimensional  sub- 
space B ^ °f  B , and  is  a linear  operator  from  H to  B^ 

of  the  form 


L 


N 


PN  L PN 


(2.6) 


Here  I’N  is  a projection  operator  of  H onto  8N  and 
f.,  = P.,f  • We  shall  assume  that  S c B M when  N < M . 

For  def initeness , we  shall  also  assume  the  initial  conditions  for 
the  approximate  equations  (2.5)  to  be  u^(0)  = P^utO)  where 
u(0)  = g (x)  is  the  initial  condition  (2.3).  Specific 
examples  of  projections  P^  and  the  resulting  approximations 
L..  will  be  given  below. 

According  to  this  general  framework,  the  formulation  of  a 
spectral  method  involves  two  essential  steps:  (i)  the  choice  of 
approximation  space  B y and  (ii)  the  choice  of  the  projection 
operator  P^  . It  will  turn  out  that  the  mathematical  analysis 
of  the  methods  also  involves  two  key  steps:  (i)  the  analysis  of 

how  well  functions  in  H can  be  approximated  by  functions  in 
B..  (see  Sec.  3)  and,  in  particular,  the  estimation  of 

I | U - PNu | | for  arbitrary  u cH;  and  (ii)  the  study  of  the 
'stability'  of  1.^  (see  Sec,  4),  Finally,  there  are  the 
important  practical  questions  of  how  to  discretize  time  (see 
Sec.  9)  and  how  to  implement  spectral  methods  efficiently  (see 
Sec.  10).  All  these  considerations  will  be  reviewed  in 
Sec.  15. 
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A 


Ga jerkin  approximation 


A Galerkin  approximation  to  (2.1-3)  is  constructed 
as  follows.  The  approximation  u^  is  sought  in  the  form  of 
the  truncated  series 

N 

u (x,t)  = }'  a (t)  A (x)  (2.6) 

N n=l  n n 

whore  the  time-independent  functions  <>n  are  assumed  linearly 
independent  and  $ e for  all  n.  Thus,  u^(x,t)  necessarily 

satisfies  all  the  boundary  conditions.  The  expansion  coefficients 
an(t)  are  determined  by  the  Galerkin  equations 


dt  ^n,uN^  ~ ^n,L  UN^  ^ ^n* f ) (n=l,...,N)  (2.7) 


or 


N da  N 

) (<>,<>)  = 7 a (A  ,LA  ) + (A  , f) 

* , vn  vm  d t L , m vn  s m yn 

m=l  m=l 


These  implicit  equations  for  an(t)  can  bc  Put  into  the 
standard  explicit  form  (2.4-5)  by  defining  the  projection 
operator  P^  by 

N N 

V(x,“  P P Pnm'V"’  Vx)  (2-S) 

n=l  m=i 

where  pjyn  are  the  elements  of  the  inverse  of  the  N x N matrix 

whose  entries  are  (A  , A ). 

rn 

Note  that  the  relation 
N N 

P,,u  = 7 J p (A  ,P„u)d>  (x) 

n-1  m=»l 

holds  for  all  projection  operators  P^.  However,  the  specific 
projection  operator  (2.8)  is  particular  to  Galerkin  approximation. 


- 13  - 


The  Galerkin  equations  (2.7)  may  bo  characterized  as  follows. 

At  each  instant  t,  we  assume  that  the  expansion  coefficients  a (t) 
in  (2.6)  are  known  and  seek  values  for  the  N independent  quantities 
(n~l,,..,N)  that  minimize 

,9un  t 3v'n 

(TT*  “ Lun'TT"  ' IjUn  • 

The  resulting  equations  for  da  /dt  are  (2.7). 

Example  2.1:  Fourier  sine  series 

If  wo  choose  H = L2(0,tt)  and  ^(x)  “ sin  nx  , we  re- 
cover the  Galerkin  approximations  given  in  Example  i.1-2  for  the 
heat  equation  and  in  Example  1.3  for  the  wave  equation.  Every 
function  uel^Ojit)  has  a Fourier  sine  series  that  converges 

in  the  L-  norm,  so  that  | |u  - P u | ] -►  0 as  N -*■ 
z N 

However,  as  illustrated  by  Example  1.3,  this  does  not  ensure 
that  the  Galerkin  approximation  converges  to  u as  N ■*  00  . 

Example  2.2:  Che  lay  s lie  v so  r i e s 

We  choose  H to  be  the  space  of  functions  on 
the  interval  |x|  < 1 that  arc  square  intograble  with  respect 
to  the  weight  function  1/  \Tl-x^  . If  the  problem  is 

ut  + u^  «=  f(x,t)  (-1  < X < 1,  t > 0)  , (2.9a) 

u(-l,t)  c 0 , u(x,0)  » g(x)  , (2.9b) 

which  is  a slight  generalization  of  Example  1.3,  it  is  appro- 
priate to  choose  the  expansion  functions  for  the  Galerkin  approxi- 
mates to  be  (x)  •»  T (x)  - (-1)  (x)  . Here  T (x)  is 

n n 0 n 

Chebyshcv  polynomial  of  degree  n definied  by  T (coaO)  **  cos  nO 

2 

when  x *=  cosO  ; thus,  T^  (x)  - 1,  T^  (x)  » x,  T^(x)  « 2x‘-l,  T.^(x) 
4x3  - 3x,...  . Observe  that  4>n(x)  satisfies  the  boundary  condition 





<>^(-1)  ■ 0 because  T^(-l)  ■ (-1)  for  all  n.  The  properties 
of  Chebyshev  polynomials  are  summarized  in  the  Appendix. 


The  Galerkin  equations  (2.7)  are  obtained  explicitly  as 
follows.  First,  the  definition  of  Tn(x)  and  the  substi- 


tution x « cos  0 imply  that 


where 


CT_ , T ) * f cos  n 6 cos  m 6d6  - xc  6 
“ 1,1  » i n nm 

0 


(f,g)  f (x)g  (x)//l-x2  dx  . 


Here  cQ  « 2,  cn  = 1 > and  5nm  * 0 n + m,  1 if 

n « m.  Therefore, 


<VV  J «nm+ 


Next,  the  Chebyshev  polynomials  satisfy 


TA+1(X)  TA-1(X) 

2 T (x)  - 

n n+1  n-1 


(n  > 2 ) , 


as  may  be  verified  by  substituting  x * cos  0.  Therefore, 


/ n (-1)  n+1m  + Trm 

- K(-l)n+1m 


n < m,  m + n odd 
n > m,  m + n odd 


n + m even 


Using  these  results,  (2.7)  gives  the  Galerkin  approximation  equations 
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T 


da  . N 

St  + *<-»*  JE  l 

m«l 


N 

“2  l P a + 
p«n+l  p 
p+n  odd 


N 

2(-l)n  l p a + f + 2 (-1) 
p-1  K 

p odd 


n 


f q ( n— If ■ 


. ,N) 


Here  £n  - (Tn,f)  tor  n - 0,...,N. 

These  Galerkin  equations  can  be  simplified  by  introducing 

N 

the  notation  Sq  ■ - £ (-1)  am,  so  that  (2.6)  becomes 

m*  1 


uN(x,t) 


N 

l 


n«0 


a ( t)  T (x)  . 
n n 


(2.10) 


Substituting  the  above  expression  for  Rq,  the  Galerkin  equations 

for  a can  be  rewritten  as 
n 


1 P + + JT“  b(t) (-l)n  (n«0,...,N),  (2.11) 

p-n+1  p n cn 
p+  n odd 


N 


I 

n*»0 


(-1)"  «. 


0 . 


(2.12) 


Here  b(t)  is  a 'boundary'  term  that  ensures  maintenance  of  the 
boundary  condition  (2.12).  Using  (2.12)  it  is  easy  to  show  that 
the  explicit  form  of  b(t)  is 


[n?0,'1,n<n!V?n’]'^Rl-i  So"1’"'"] 
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b (t) 


-1 
— c 


Tau  approximation 


The  tau  method  was  invented  by  Lanczos  in  1938  (see  Lanczos 
1956).  First,  the  expansion  functions  4>n  (n-1,2,...)  are 

assumed  to  be  elements  of  a complete  set  of  orthonormal  functions. 
The  approximate  solution  u^x.t)  is  assumed  to  be  expanded  in 
terms  of  those  functions  as  in 


UjjfXrt) 


N+k 


I «n(t)*n(x) 

n«l 


(2.13) 


Here  k is  the  number  of  independent  boundary  constraints  BuN“° 
that  must  be  applied.  The  important  difference  between  (2.13) 
for  tau  approximation  and  (2.6)  for  Galerkin  approximation  is  that 
the  expansion  functions  <)>n  in  (2.13)  are  not  required  individually 
to  satisfy  the  boundary  constraints  (2.2).  The  k boundary 
constraints 


N+k 

l an  B *n  " 0 (2.14) 

n»l  " 


are  imposed  as  part  of  the  conditions  determining  the  expansion  co- 
efficients of  a function  in  8^.  Then,  the  projection  operator 
PN  is  defined  by 


N 


n*l 


Vn 


+ 


k 

bm^N+m 

m«l 


(2.15) 
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T ••'•'T’ 


bppph — — — *mmm 


where  b (m=l,...,k)  are  chosen  so  that  the  boundary  con- 
m 

straints  are  satisfied:  BPNu  = 0 for  all  u e >^ . 

It  follows  from  these  definitions  that  the  tau  approximation 
to  (2.1-2)  is  given  by  (2.13)  with  the  k equations  (2.14) 
and  the  N equations 


dan 

dt“  = (*n'L  UN)  + (*n'f)  (n-l,...,M)  . (2.16) 


An  equivalent  formulation  of  the  tau  method  is  given  as 
follows:  The  equations  for  the  expansion  coefficients  aQ  of  the 
exact  solution  u in  terms  of  the  complete  orthonormal  basis  $ are 

oo 

u(x,t)  = l a ( t ><J>  (x)  , 
n-1  n n 


da 

St~  “ ^n'Lu*  + (n*l,  2, . . . ) . (2.17) 

The  tau  approximation  equations  for  the  N+k  expansion  co- 
efficients of  Ujj  in  (2.13)  are  obtained  from  the  first  N 
equations  (2.17)  with  u replaced  by  uN  and  the  k boundary 
conditions  (2.14). 

The  origin  of  the  name  ’tau  method’  is  that  the  resulting 
approximation  uN  is  the  exact  solution  to  the  modified  problem 


3u 

W 


N 


L u 


N 


* + l 

P=1 


TP(t,Vp(*> 


(2.18) 
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which  lies  in  £ for  all  t > 0 . For  each  initial  value  problem 
N 

and  choice  of  orthonormal  basis  4>n  (and  associated  inner  product)  , 
there  is  a (normally  unique)  choice  of  x-coef f icients  such  that 

UN  e *’N  ' n««»ely 


Tp  ■-  ‘♦N+p’LuH  + f) 

The  remaining  tau  coefficients  x ^ , x2,...,  x^ 
the  k boundary  constraints 


B 


3un 

TT 


- o 


and  the  N dynamical  constraints  (2.17)  for  n 


( p - k+1,  k+2,  . 

are  determined  by 


lf.a.fN. 


) 


Example  2.3:  Fourier  sine  series 


For  all  of  the  applications  given  in  Example  2.1,  Galerkin 
and  tau  approximations  based  on  * /~jp  sin  nx  are  identical 

(except  for  the  scaling  factor  *'2/ it  ) since  the  orthonormal 

expansion  functions  <f>n  satisfy  the  boundary  conditions. 


Example  2.4:  Chebyshev  series 

If  we  choose  ^n+i^x^  ~^c  Tn*x*  where  cQ  = 2,  cfi  = 1 
(n  > 0)  and  apply  the  tau  method  to  the  problem  (2.9)  the  result 
can  be  recast  into  the  form  of  equations  (2.10-12)  with  b(t)  = 0 
and  (2.11)  only  applied  for  n = 0,1,...,N-1  instead  of 
n * 0,1,..., N.  Thus,  the  tau  equations  for  the  one-dimensional 

wave  problem  (2.9)  are  (2.10)  with 


dan 

St 


N 

— Z pa+f  (0  < n < N-l ) 
cn  p-n+1  P n 
p+n  odd 


(2.19) 


N 

Z (-l)n  a ( t) 
n-0  n 


In  this  problem,  /p  Xj(t)*  while  xp(t) 


(2.20) 
for  p > 1 
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Example  2.5:  Laguerre  series 


Here  we  choose  V to  be  the  space  of  functions  that  are 
square  integrable  on  0 < x < ® with  respect  to  the  weight 

» Y 

function  e . We  choose  the  expansion  functions  to  be 

$ (x)  * L (x)  where  L (x)  is  the  (normalized)  Laguerre  poly- 
n n n 

nomial  of  degree  n:  LQ(x)  = = 1-x, 

L^(x)  * 1 - 2x  + ^ x2,  ...  . 

Suppose  we  wish  to  solve  the  problem 

ufc  + u^  = f(x,t)  (0  £ x < «>,  t > 0)  (2.21a) 


u ( 0 , t)  = 0 , 


u (x , 0) 


g(x) 


(2.21b) 


by  seeking  an  approximate  solution  of  the  form 


uN(x,t) 


N 

E a (t)L  (x) 
n n n 
n=0 


(2.22) 


To  derive  the  tan  equations  for  a (t)  , we  note  that  Ln(x) 


satisfies  L (0)  =1,  L'  - L'  , 
n n n+i. 


L t n — 0 r 1 , • • • 
n 


and 


(L  ,L  ) = L ( x ) L ( x ) e x dx 
n m On  m 

(2.17)  is 


5 . Thus,  the  tau  approximation 

nm 


da 


n 


dt 


E a + (Ln,f] 
p-n+1  * 


(n  = 0,1, . . . ,N-1) 


(2.23) 


while  the  boundary  condition  is 


N 

E a. 

n=0 


n 


0 


(2.24) 
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Similarly,  the  Laguerre-tau  approximation  to  the  heat 


equation  problem 


ufc  - uxx  + f(x,t)  (0  £ x < t > 0) 


u(0,t)  - 0 u(x,0)  * g (x) 


is  given  by  (2.22),  (2.24)  and 


(2.25) 


da  N 

g-B  * l (p-n-l)ap  + (Ln,f)  (n-0,1, . . . ,N-1) 
p=n+l 


(2.26) 


Collocation  or  pseudospectral  approximation 

The  projection  operator  PN  for  collocation  {sometimes 
called  the  method  of  selected  points  (Lanczos  1956)  or  pseudospectral 
approximation  (Orszag  1971c)]  is  defined  as  follows.  Let 
Xj.Xj,...,^  be  N points  interior  to  the  domain  D.  These 
points  are  called  the  collocation  points.  Also  let  $ (x) 

T\ 

(n«l,...,N)  be  a basis  for  the  approximation  space  8 and 
suppose  that  det  A (x  ) / 0.  Then  for  each  u r H 

n in 


P u = l a <f>  (x) 
N n n 

n*  1 


(2.27) 


where  the  expansion  coefficients  an  are  the  solutions  of  the 
equations 


- 21- 


(2.28) 


N 

l an  ^{x^  » u(Xj)  (i=l,...,N) 


Thus,  collocation  is  characterized  by  the  conditions  that 
PMu(x.)  * u(x.)  for  i * 1, . . . ,N  and  P..ue  8 . Notice  that 

the  results  of  collocation  depend  on  both  the  points  xn  and 
the  functions  <}>n(x)  for  n = !#•••»**  • 

Example  2.6:  Fourier  sine  series 

If  we  wish  to  solve  the  problems  formulated  in  Examples 
1.1-3  by  collocation  instead  of  Galerkin  or  tau  methods 
we  proceed  as  follows.  We  choose  the  space  = L2(0,tt), 
the  expansion  functions  4>n(x)  = sin  nx  (n=l,...,N),  and  the 
collocation  points  x^  = 7rj/(N+l)  (j=l,...,N).  The 

collocation  equations 

N 

l an  sin  S+T  = U(xi)  (-j=l,...fN>  (2.29) 

n=l  J 

have  the  explicit  solution 

N 

2 

an  = N+T  u(xj)sin  S+T  "(n=1'*-*'N)  (2.30) 


This  result  follows  from  the  relation 


N 


l 

n=l 


sin 


sin 


nkn 

N+l 


N+l  x 
2 6 jk 
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valid  for  0 < j,k  < N+l  . Thus, 


N 


PNu  = l an  sin  nx 
n*l 


(2.31) 


where  an  is  given  by  (2.30). 

It  follows  from  (2.29-31)  that 


pnlpnu 


N 

7 b sin  nx 
*•  n 

n=l 


where  b_  = - n^  a (n=l,...,N)  if  L = 3^/3x^  , and 
n n 


n 


N 

l 


tt  n 
m sin  N+T 


’ — 

N+l  Trm  Trn 

m-1  cos  N+l  • cos  n+T 
m+n  odd 


Trn  m 


a ( n 1 , . . . ,N) 


if  L a 3/3x  . 

Example  2.7:  Chebyshev  collocation  for  the  wave  equation 

Suppose  we  wish  to  solve  the  one-dimensional  wave  problem 
(2.9)  using  collocation.  An  appropriate  basis  for  the  approxi- 
mation space  bn  is  the  set  of  functions  0n(x)  = Tn(x)  “ (-l)nTQ(x) 
(n*l,...,N)  introduced  in  our  discussion  of  Example  2.2  above. 

We  choose  the  collocation  points  to  be  the  extrema  of  the 

Chebyshev  polynomial  TN(x)  satisfying  |x|  < 1 . Since 

T„ (cos  0)  ■ cos  N0  , these  extrema  lie  at  x.  * cos  ^ for 
N D N 

j = 0 , . . . , N- 1 . The  point  x^  ■ - 1 is  also  an  extremum  of 
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T^(x)  but  it  is  npt  included  in  the  set  of  collocation  points 

because  the  boundary  conditions  for  (2.9)  are  imposed  at  x - -1 

so  *n(-l)  - 0 for  all  n . 

As  in  Example  2.2,  the  expansion  coefficients  an  for 

n - 1,...,N  may  be  augmented  by  defining  an  ■ - f (-l)m  a 

o m 

so  that  m“1 

N 

uN(x,t)  - l an(t)T  (x)  . 
n-0 

It  may  then  be  shown  that  the  collocation  equations  for  a (E) 
that  follow  from  (2.9)  are 


dan 

cTt“ 


N 

" i P ap  + fn  + JT  L(t)  <"1)n  <"-<>*• 

n p«n+l  v “ cn 
p+n  odd 


N 


• # N)  (2.32) 


l (-Dn  an(t)  - 0 
n-0 


(2.33) 


where  fn  - (Tn,f)  and  cQ  - SN  - 2 , 5n  - 1 (0  < n < N) . 

Here  b(t)  is  a 'boundary*  term  that  is  used  to  ensure  compliance 
with  the  boundary  condition  (2.33).  It  may  also  be  shown  that 


b (t) 


“ ^ l ("l)n(n*a  +?  ) - I 1 

n-0  n n N ^ D *nJ 
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The  reader  should  observe  the  close  similarity  between  the 
Chebyshev  Galerkin,  tau,  and  collocation  equations  for  the  problem 
(2.9).  The  only  difference  between  them  is  the  way  the  boundary 
term  b(t)  enters.  In  the  Galerkin  equations  (2.11),  b(t)  appears 
with  the  coefficient  (-l)n/cn?  in  the  tau  equations  b(t)  enters 
with  the  coefficient  6rN  so  it  appears  only  in  the  equation  for 
aN  as  a tau  coefficient;  with  collocation,  the  coefficient  of 
b(t)  is  (-l)n/cn.  This  close  similarity  between  the  three  methods 
for  the  wave  equation  can  also  be  seen  by  observing  that  when 
f(x,t)  is  a polynomial  of  degree  N in  x,  all  three  approxi- 
mation methods  give  Nth  degree  polynomial  approximations  uN(x,t) 
that  satisfy  exactly  the  initial-boundary  value  problem 


3t“  + lx-  “ f *x,t)  + T 


(2.34) 


uN(-l, t) 


In  the  tau  method,  Q„(x)  = T„(x);  in  collocation, 

N N 

N-l  , M N / i \ n+N  -i  i N 

Qn  (x)  - n (x-x.)  - 2l  N [ T (x)  - i 2i_N(x-l)T' (x) 

N j-0  j \ c n N N 

J n*0  n 

where  x^  ■>  cos  (j  = 0,...,N-1)  are  the  collocation  points; 

finally,  the  Galerkin  equations  (2.10)  are  obtained  if 


V*'  - l 

n«0  n 
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For  all  three  methods  t (t)  is  uniquely  determined  by  the 
requirement  that  u^Xpt)  be  a polynomial  of  degree  N in  x 
that  satisfies  the  boundary  condition  u^(-l,t)  = 0 for  all  t . 

Example  2.8:  Chebyshev  spectral  methods  for  the  heat  equation 
To  illustrate  further  the  nature  of  the  differences  between 
Galerkin,  tau  and  collocation  methods,  we  apply  them  to  the 
heat  equation 


§£  = + f (X,t)  (-1  < x < 1 , t > 0), 

d X 


u(-l,t)  = u (1 , t)  =0  (t  > 0),  u (x,  0)  * g{x)  (-1  < X < 1). 


We  approximate  u(x,t)  by 


u^x^)  = l an(t)Tn(x)  . 
n*0 


The  Galerkin,  tau,  and  collocation  equations  for  an(t.)  are  a11 
of  the  form 


i l p(p2-n2)ii  +f  (t).b  (t)Bln+b2(t)B2n 

n p«=n+2  K 

p+n  even 


(2.35) 


N N 

l \ - ! i-D  - o 

n=0  n=0 


(2.36) 
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whore  f = (T  ,f).  Eqs.  (2.36)  are  just  a restatement  of 
n n 

u„  ( 4 1 , t ) = 0.  The  terms  b.  (t)  and  b.,(t)  in  (2.3b)  are 
N Id! 

boundary  terms  that  ensure  compliance  with  tlie  boundary  condi- 
tions (2.36).  The  only  differences  between  the  three  approximation 

methods  lies  ’ ^ the  coefficients  R,  and  B~  . 

In  2n 

In  the  tau  method. 


Bln  ‘ 6n,N-l  ' R2n  " 6nN 


(2.37a) 


In  the  Galerkin  method, 


In 


B 


(-1) 


n 


2n 


c 


(2.37b) 


this  result  follows  using  the  expansion  functions 


<t>  (x)  = T (x)  - 
Yn  n 


Tq(x)  n even 

^ T.  (x)  n odd 


that  satisfy  <J>  ( + 1)  - 0 and  augmenting  the  expansion  coef- 


ficients an  for  n > 2 by  an  = - J 


2n 


and 


•i  - - l 


Finally,  with  collocation  performed  at  tlie  points  x ^ - cos  N 


2n+ 1 * 

n j 


(j  = 1,2,...,N-1)  the  coefficient: 
are  given  by 


B,  and  Rn  in  (2.35) 
In  2n 


B 


1 

In  c 7 
'-n 


n 


2n 


_ (-D 


n 


(2.37c) 


It  may 


ay  also  be  verified  that  the  boundary  terms  ^ (t) 


and  b2(t)  are  of  the  form 


bi(t)  - c.+ 


.2  N 

Mi  + l fn 

3x‘U+l  nc° 


+ V 


2 

3x 


N 


L-5|  + l 

XB-l  -0 


('.’.38) 
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2+ 


(-D 

N2+N 


N 


2- 


N+§ 

N2+N 


for  the  Galerkin  method; 


1+ 


1 

N 


1- 


= 0 


c =n 
2+  u 


c 

2- 


1 

N 


for  the  collocation  method. 


In  the  previous  examples  the  only  difference  between  Galerkin, 
tau,  and  collocation  approximations  is  their  treatment  of  the  boundary 
terms.  However,  in  more  complicated  problems,  there  are  significant 

differences  between  these  approximations . The  next  example  illustrates 
the  influence  of  quadratic  nonlinearity. 

Example  2.9:  Chebyshev  approximations  to  Burgers'  equation 
Chebyshev  series  approximations  to  the  solution  u(x,t)  to 
Burgers'  equation 


+ u = 
3t  u 3x 


A 


(|x|*i,t>0) 

9x2 


(2.39) 


u(±l,t)  = 0 
u(x,0)  = f (x) 


are  obtained  by  methods  very  similar  to  those  for  linear  equations. 
In  general,  spectral  approximations  to  the  nonlinear  equation 


= A (u) 


(2.40) 


are  of  the  form 


TIT  * rNAIV'N» 


(2.41) 


where  is  a projection  operator.  The  projection  operator 

P can  be  that  for  Galerkin,  tau,  or  collocation  approximations, 

N 

If  we  write 


UN  (>:,t) 


= / Vt)Tn(x>' 

n-0 


then 


the  Galerkin  approximation  to  (2.39)  is  given  by 


2 I p am  + v l m(m2-n2)am  + b+(t)  + b_(t)  (-l)n 

ml <N  m=n+2 

m+n  even 

p I <N  (0<n<N)  , 


| m | <N 


m+pin+1 
n+m+p  odd 


(2.42a) 


* Observe  that  if  (u,  Au)  = 0 so  the  system  (2.40)  lias  the 
energy  integral  3(u,u)/3t  = 0,  then  (2.41)  has  the  energy 
integral  3(u^,uN)/3t  = 0 provided  that  the  projection  operator 

PN  is  self-adjoint.  This  follows  from 

= (IWA(IW  * = °* 

An  example  of  a self-adjoint  projection  operator  P in  the 

Galerkin  operator  (2.8).  Energy  conservation  is  guaranteed 
only  if  the  inner  product  used  in  the  definition  of  the  Galerkin 
approximation  is  the  same  as  that  in  the  energy  integral. 
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■""""T  - 


N 


n~-0 


N 


I an(-l)n  = 0 , 
n=0 


(2.42b) 


whore  am  — ° | m | a|,n|  for  M ^ N.  The  tau  equations  are  identical 
except  that  (2.42a)  only  applies  for  0 £ n £ N - 2 and  b+  = b = 0. 
On  the  other  hand,  the  collocation  equations  obtained  using  the  collo- 
cation points  Xj  = cos  for  j = 1,...,N-1  are  just  (2.42b)  and 


5 f!" 

n dt 


= “ 2 l P 
| m | <N 

|P|<N 


a a 
m p 


-2  l P 
|m|  <N 
I Pi  <N 


a a 
m p 


m+p>n+l  m+p>2N-n+l 

n+rn+p  odd  n+m+p  odd 


N 

+ v l m(m2-n2)am  + b+  (t)  + b_(t)(-l)n 
m=n+2 


m+n  even 


(2.43) 


(O^n-N) 

where  c^  - c^  = 2 and  cn  = 1 for  n f 0,N.  Observe  the  appear- 
ance of  the  'aliasing'  term  as  the  second  sum  on  the  right  side  of 
(2.43).  Aliasing  is  discussed  in  detail  by  Orszag  (1971a,  1972). 

Example  2.10:  Chebyshev  approximations  to  u + F(u)  =0 

- w X 

Galerkin  and  tau  approximations  to  the  solution  to 

Ut  + F(u)x  a 0 (2.44) 

where  F(u)  is  arbitrarily  nonlinear,  are  very  unwieldy  both 
to  write  down  explicitly  and  to  solve  on  a computer.  On  the  other  hand 
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E ■ . - 

while  the  collocation  equations  may  also  be  hard  to  write  down 
explicitly,  they  lend  themselves  to  ready  solution  without  their 
explicit  form  being  known! 

The  collocation  approximation  to  (2.44)  is  obtained  as  follows. 
We  use  the  relation 

3un 

(F<V>x  - F’<V  sr  • (2-45) 


( 


Since  Su^/dx  can  be  computed  explicitly  in  terms  of  u^  as  a poly- 
nomial in  x of  degree  N-l,  it  follows  that  (Flv^)^  can  be 
evaluated  by  this  formula  at  each  of  the  collocation  points  assuming 
that  F'  ( z ) is  a known  function;  thus,  the  collocation  approxima- 
tion to  (2.44)  is  determined. 

There  is  a slightly  different  collocation  procedure  that  can  also 
be  applied  to  (2.44).  It  has  the  operator  form 

5T1  ♦ PN  h PN  F‘V  - 0 ■ <2-46> 


which  is  usually  not  the  same  as  the  collocation  approximation  of 

the  form  (2.41)  described  above.  In  this  approximation,  9u  /at  is 

N 

ocnputed  by  first  using  collocation  to  obtain  P^Ffu^)  from  arri  then 

using  the  collocation  approximation  PN3/9x  to  a/3x  given  in  Example  2.7. 
The  collocation  approxiration  given  by  (2.41)  or  (2.45)  differs  from 
(2.46)  by  the  term 


N 


3x  (I-PN) 


F(V 


which  is  generally  not  zero.  However,  if  F'  (z)  is  not  known 
accurately  then  (2.46)  may  be  the  only  viable  method. 
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3.  Survey  of  Approximation  Theory 

The  remarkable  convergence  properties  of  spectral  methods  to 
be  discussed  later  owe  to  the  rapid  convergence  of  expansions  of 
smooth  functions  in  series  of  orthogonal  functions.  We  present 
a summary  of  the  relevant  theory  here. 

Fourier  series 

The  complex  Fourier  series  of  f(x)  defined  for  0 - x - 2u 
is  the  periodic  function 

g (x)  = l a.  eiKx  (3.1) 

k=-«  K ' 


where 


/2  f(x)e’lkxdx  . 
0 


(3.2) 


We  shall  show  below  that  if  f (x)  is  piecewise  continuous  and  has 
bounded  total  variation  then 

g (x)  = | [f (x+)+f(x-)]  (3.3) 

for  0 - x - 2tt  and  g(x)  is  repeated  periodically  outside  the  interval 
0 < x « 2it.  In  particular,  g(0)  * g(2ir)  ■ $ If (0+)+f l2n-) ] . 

The  Fourier  sine  series  of  a function  f(x)  defined  for 
0 < x < it  is  the  function 

CO 

g_(x)  * l av  sin  kx  , (3.4) 

8 k-1  K ' 
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where 


i * 

a.  * — / f(x)  sin  kx  dx  . (3.5) 

k tt  Q 

The  Fourier  cosine  series  of  a function  defined  for  0 < x < tt  is 

00 

9CU)  = l ak  cos  kx  (3.6) 

k=0 

where 

a,  = — =—  / f (x)  cos  kx  dx  (3.7) 

k ”Ck  0 

with  c -2,  cv  * 1 (k  > 0) . It  follows  easily  from  (3.3)  that  if 

f(x)  is  piecewise  continuous  and  has  bounded  total  variation  then 

gs(x)  = fs(x> f (3,8) 

gc(x)  ■ fc(x)/  (3'9) 

where  f (x)  = f„  (x)  = § [f  (x+)+f  (x-) ) for  0 < x < it, 
s c 

f8(-x)  = -fg(x),  fc(-x)  = fc(x)  for  -TT  < x < 0,  fs(0)  = fg(lT)  = 0, 
fc  (0)  = f(0+),  fc(Tr)  = f(Tr-),  and  fg(x)  and  fc  ^x^  are  extended 
periodically  outside  the  interval  - it  < x - tt. 


Convergence  of  Fourier  series 
To  prove  (3.3)  we  define 

gK(x) 


g (x)  as  the  partial  sum 

Iv 


K 


l ave 


ikx 


k*=-K 


(3.11) 


Using  (3.2)  and  the  trigonometric  sum  formula 


sin [ (K+i )•! 


l eiks 

k=-K 


sin  (is) 


we  obtain 


gK(x) 


1 

I? 


x 


/ 

x-2n 


sin( (K+i) t) 
f(x-t)dt 


sinlit] 


(3.12) 


The  kernel  sin(K+i )t /sin  it  of  the  integral  (3.2)  is 
plotted  for  several  values  of  K in  Fig.  3.1.  This  plot 
suggests  that  when  f(x)  has  bounded  total  variation  the  leading 
contribution  to  the  integral  as  K -*  00  comes  from  the  neighbor- 
hood of  t = 0 since  the  contributions  from  the  rest  of  the  in- 
tegration region  should  nearly  cancel  due  to  the  rapid  oscillations 
of  the  integrand.  Thus, 


+e  sin[(K+|)t] 

g*(x)  x TZ  I f (x-t)dt  (K*») 

K ' sin  lit) 

• ^ 


(3.13) 


for  any  fixed  e > 0.  Since  e may  be  chosen  small  we  may  replace 
sin  it  by  it  with  a maximum  error  of  0(e).  Also  since  f(x-t) 
is  piecewise  continuous,  we  may  assume  that  f (x-t)  is  continuous 


for  0 - t - c and  -c 

1 t < 0 with 

at  worst  a jump 

discontinuity 

at  t * 0. 

Therefore  we 

may  replace 

f(x-t)  by  f(x-) 

for  t > 0 

and  f(x+) 

for  t < 0 

giving 
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1 e sin(K+$)s 

gR(x)  ^ [f  (x+)+f  (x-)  ] i / ds 


(K-*») 


Since 


i e sin(K+i)s 
— / ds 

TT  } 3 


x (K+i)e 
tT  1 


sin  s 
s 


ds  ^ / 

7T  J 


1 /■  sin 


ds  = i (K-*°°) 


for  any  fixed  e > 0,  we  obtain 


gR(x)  'v-  itf  (x+)+f  (x-)  ] 


proving  (3.3). 


In  the  neighborhood  of  a point  of  discontinuity  of  f (x) 
(or  x = 0 and  x = 2tt  if  f (0+)  ^ f(2ir-)]  the  convergence 
of  gR(x)  to  its  limit  (3.3)  as  K -*■  00  is  not  uniform.  To 
investigate  the  detailed  approach  of  gR(x)  to  g(x)  near  a 
point  of  discontinuity  xQ  of  f (x) , we  use  the  asymptotic 
integral  representation  (3.13)  to  obtain 


,6  sin[(K+i)t]  _ 

W ^r>  ' il  t '<V-  -*>« 


for  every  fixed  z.  Since  e is  assumed  small  we  can  approxi- 
mate f (x0+s)  by  f(xQ+)  for  0 < s < e and  by  f(xQ-)  for 
-e  < s < 0.  Therefore,  for  each  fixed  z and  c, 
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..W-I.JI1WI  I -T-  . .....  r^.  - , . , ^ _ ■ I - - ' H 


f(xn+)  z/(K+i)  8in(K+i) 


-*-k0T/  . Bi'MRTj;  f (xn-)  e eJ«/K4.Hf 

W rr'  ' — f-  I — — “ * -r-  / sin(*  * 

. RTj  “c.  t 


2/ (K+i ) 


(K+®) 


f (xQ+) 


z f(x  -)  e(K+*) 

/ sin_s  dg  + 0_  , sin_s  dx 

* S IT  ' 


IT  • S 

-e (K+i) 


* !££L  J*  ax  + i!Vl  5iS_®  as 

TT  1 S TT  * 


S 


(K-*-®) 


Since  / sin  s/s  ds  = it,  we  obtain 


gR(x0+  — ) -v  ilf  (x0+)+f  (x0-)]  + £[f  (xQ+)-f  (x0-)]  Si  (z) 
K+i 


<K-®) 


(3.14a) 

for  any  fixed  z.  Here  the  sine  integral  Si(z)  is  defined 

Si(z)  = | /2  2±!L-£  ds  (3.14b) 

TT  0 S 

A plot  of  Si(z)  is  given  in  Fig.  3.2. 

The  result  (3.14)  shows  that  if  x - xQ  = 0 (^)  as  K -*■  ® then 
9K(x)  ~ £ tf (xg+) +f (xo“) 1 ™ 0(1).  This  shows  the  nonuniformity  of 
convergence  of  gR(x)  to  f (x)  in  the  neighborhood  of  the  discon- 
tinuity Xq.  This  nonunifarm  behavior  of  the  limit  gR(x)-+f(x)  as  K -*•  ® 
is  called  the  Gibbs  phenomenon. 

lb  illustrate  the  Gibbs  phenomenon  in  an  actual  Fourier  series,  we  plot 
in  Fig.  3.3  the  partial  sums  of  the  Fourier  sine  series  expansion 
of  the  function 


f(x)  - x/tt  (0<x<tt ) 

The  extended  function  fg(x)  is  discontinuous  at  x = n leading 
to  the  Gibbs  phenomenon  there. 
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Fig.  3.3.  A plot  of  the  N-term  partial  suns  of  the  Fourier 
sine  aeries  expansion  of  the  function  x/ff  for  N“10,20,40. 
The  function  x/ir  is  also  plotted . Observe  the  Gibbs  phenoa 


As  K -*•  •»,  the  maximum  error  of  the  partial  sums  of  a Fourier 
(complex  or  sine  or  cosine)  series  in  the  neighborhood  of  a point 
of  discontinuity  occurs  at  the  maximum  of  Si(z)  . 

Since  Si’(z)  » 0 when  z - nn  for  n ■ ±1,±2,...,  the  maximum 
error  must  occur  at  one  of  these  points.  It  is  easy  to  argue  that 
the  maximum  of  Si(z)  actually  occurs  at  z ■ u where 

- Si  (it)  * .58949  (3.15) 

2 

Thus  the  maximum  overshoot  of  the  partial  sums  of  the  Fourier 

series  near  a discontinuity  occurs  near  x = xn  + — — for  K 

° K+J 

large  and  is  of  magnitude 


gK 


(x 


- f(xQ+)  * .08949[f (xQ+)-f  (x0-)J 


(K-°°) 


(3.16) 


where  the  quantity  in  square  brackets  is  the  jump  at  xQ.  For  the 
example  plotted  in  Fig.  3.3  the  jump  of  ffi(x)  at  x = it  has 
magnitude  2 so  the  Fourier  series  gives  a local  overshoot  of 
magnitude  0.179. 

As  z •*  + «>,  Si  (z)  -*•  ± 1 so  that  (3.14)  is  consistent  with 

the  convergence  of  the  Fourier  series  to  f (xQ+)  just  to  the  right 
of  Xq  and  to  f(Xg-)  just  to  the  left  of  Xq.  The  Gibbs  phenomenon 
only  appears  when  x x0  at  the  rate  1/K  as  K <*>. 

Rate  of  Convergence  of  Fourier  Series 

If  f (x)  is  smooth  and  periodic,  its  Fourier  series  does  not 
exhibit  the  Gibbs  phenomenon.  The  Fourier  series  of  such  an  f(x)  con- 
verges rapidly  and  uniformly.  Suppose  f(x)  is  periodic  and  has 


-40- 


I 


continuous  derivatives  of  order  p = 0,1,..., n-1  and  f^(x)  is 
integrable.  Applying  integration  by  parts  to  (3.2),  it  follows  that 


a 


k 


1 

2n  (ik) n 


f(n) (x)e-ikx 


dx 


Since  f^(x)  is  integrable,  the  Riemann-Lebesque  lemma 
implies  that 


ak  « l/kn  (k  - ±«)  . (3.17) 

Note  that,  because  f(x)  is  periodic,  continuity  of  f^(x) 

also  requires  f^(0)  = f^p^(2tr)  . It  follows  from  (3.17) 

that  if  f (x)  is  continuous  with  f(0)  = f(2n)  and  f ' (x)  is 

integrable  then  <<  1/k  as  k -►  °°  ; if , in  addition,  f ' (x) 

2 

is  piecewise  continuous  and  differentiable  then  ak  = 0(l/k  ) 
as  k » . 


Now  we  can  be  more  precise  in  our  estimates  of  the  error 
gR(x)  - f(x)  . If  a^  goes  to  zero  like  l/kn  as  k -*•  « 
and  no  faster,  then  f^n_1^(x)  is  discontinuous.  In  this  case. 


%(x) 


f (x)  = 0(-L)  (K  ■+•  “>  ) 
K 


(3.18) 


when  x is  fixed  away  from  a point  of  discontinuity  of  f^n_1^ 
as  K -*•<*>  , while 

gK(x)  - f(x)  = 0(-^-)  (K  - oo)  (3.19) 

K 

when  x - Xq  = 0 (^)  as  K -*■  00  where  xQ  is  a point  of 
discontinuity  of  f ^ (x) . 

In  particular,  if  f (x)  is  infinitely  differentiable  and 
periodic  [f(x+2ir)  * f (x)  ] , gR(x)  converges  to  f (x)  more 
rapidly  than  any  finite  power  of  1/K  as  K -*•  <*>  for  all  x . 


Fourier  sine  and  cosine  series  have  convergence  properties 
very  similar  to  the  complex  Fourier  series  just  discussed.  We 
summarize  these  properties  for  Fourier  cosine  series.  If  deri- 
vatives of  f(x)  of  order  p = 0,1,..., n-1  are  continuous  for 
0 < x < it  while  f *p*  (0)  = f ^p*  (tt)  = 0 for  all  odd  p < n 
and  f (n)  (x)  is  integrable,  then  the  Fourier  cosine  coefficients 
given  by  (3.7)  satisfy 

an  <<  l/kn  (k-®)^  (3.20) 

as  may  be  proven  by  integration  by  parts. 

Thus,  if  f (x)  is  infinitely  differentiable  for  0 <_  x <_  it 
and  f^2p+1^(0)  * f^2p+1^(rr)  = 0 for  p = 0,1,...  then  the 
Fourier  cosine  coefficients  approach  zero  more  rapidly 

than  any  power  of  1/k  as  k + ® . In  other  words,  if  f (x) 

is  infinitely  differentiable  on  -®  ^ x <_  ®,  periodic  with  period 
2tt  [f(x+2rr)  = f(x)]  , and  even  [f(x)  = f (-x)  ) , 
then  the  remainder  after  n terms  of  the  Fourier  cosine  series 
(3.§)  goes  to  zero  more  rapidly  than  any  finite  power  of  1,A| 
as  K -*•  ® 

To  compare  the  convergence  properties  of  Fourier  sine  and 

cosine  series,  we  have  plotted  in  Figs.  3.3  and  3.4  some  results 

for  the  Fourier  sine  and  cosine  expansions,  respectively,  of  the 

function  x/tt  for  0 < x < n . As  discussed  above,  the  Gibbs 

phenomenon  in  the  sine  series  expansion  is  evident  at  x = tt  (see 

Fig.  3.3).  Observe  that  the  error  in  the  N term  partial  sum 

goes  to  zero  like  1/N  as  N -*■  ® when  x is  fixed  0 <_  x < tt  • 

The  Gibbs  phenomenon  near  x = tt  slows  the  convergence  of  the 

Fourier  series  for  all  x.  In  Fig.  3.4,  we  plot  the  error  between 

the  N term  cosine  series  and  x/tt  . Observe  that  as  N -*■  ® the 

2 

error  goes  to  zero  like  1/N  for  0 < x < tt  and  like  1/N  when 
x * 0 ( 1/N)  as  N -*■  °°  . 
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025 


1 ■ 


v polynomial  expansions 


The  convergence  theory  of  Chebyshev  polynomial  expansions 
is  very  similar  to  that  of  Fourier  cosine  series.  In  fact,  if 


g(x)  = l akTk(x) 
k=0 


(3.21) 


is  the  Chebyshev  series  associated  with  f (x)  for  -1  <_  x £ 1 
then  G(e)  = g(cos  0)  is  the  Fourier  cosine  series  of 


F (0 ) = f (cos  0)  for  0 < 0 < it 


This  result  follows  from 


the  definition  of  T (x) : since  T (cos  0)  = cos  n 0, 

n n 


G (0 ) « g(cos0)  = l an  cos  n0  . Thus, 

k=0 


/ f(cos0)cos  k0  d0  = 

"ck  0 *Ck 


/ f (x)T.  (x)  (1-x2)  dx 
-1 

(3.22) 


where  cQ  = 2 , c^  = 1 (k  > 0 ) . 

It  follows  from  this  close  relation  between  Chebyshev 
series  and  Fourier  cosine  series  that  if  f (x)  is  piecewise 
continuous  and  if  f (x)  is  of  bounded  total  variation  for 
*1  f x < 1 then  g(x)  * -j-tf  (xt) +f  (x-)  ] for  each  x (-1  < x < 1) 
and  g (1)  = f(l-),  g(-l)  = f(-l+)  . Also,  if  f (p) (x)  is 
continuous  for  all  |xj  £ 1 for  p = 0,l,...,n-l,  and  f ^ (x)  is 

integrable,  then 


ak  <<  1/k 


(3.23) 
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it  follows  that  the  re- 


Since  iT^x)  | < 1 for  |x|  <1  , 

mainder  after  K terms  of  the  Chebyshev  series  (3.23)  is  asynptotically 
much  smaller  than  1/Kn  * as  K -*■  «.  If  f (x)  is  in- 
finitely differentiable  for  |x|  <1  , the  error  in  the 
Chebyshev  series  goes  to  zero  more  rapidly  than  any  finite 
power  of  1/K  as  K • 

The  most  important  feature  of  Chebyshev  series  is  that 
their  convergence  properties  are  not  affected  by  the  values 
of  f(x)  or  its  derivatives  at  the  boundaries  x « ±1  but 

only  by  the  smoothness  of  f (x)  and  its  derivatives  throughout 
-1  < x < 1 . In  contrast,  the  Gibbs  phenomenon  shows  that  the 
rate  of  convergence  of  Fourier  series  depends  on  the  values  of 
f and  its  derivatives  at  the  boundaries  in  addition  to  the 
smoothness  of  f in  the  interior  of  the  interval.  The 
reason  for  the  absence  of  a Gibbs  phenomenon  for  the  Chebyshev 
series  of  f (x)  and  its  derivatives  at  x * ± 1 is  due  to  the  fact 

that  F (8 ) - f(cos  9)  satisfies  F(2p+1*(0)  ■ F^2p+1)U)  * 0 
provided  only  that  all  derivatives  of  f(x)  of  order  at  most  2p+l 
exist  at  x - ± 1. 

An  important  consequence  of  the  rapid  convergence  of  Chebyshev 
polynomial  expansions  of  smooth  functions  is  that  Chebyshev  expansions 
may  normally  be  differentiated  termwise.  Since 

T (x)  - 0 (k2p)  (k  + ») 

dxp 

uniformly  for  |x|  < 1 [see  Appendix],  if  -►  0 faster  than  any 
finite  power  of  1/k  as  k -*■  ® then  (3.21)  implies 
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(as  may  be  proven  by  an  elementary  uniform  convergence  argument) . 

While  Chebyshev  expansions  do  not  exhibit  the  Gibbs  phenomenon 
at  the  boundaries  x = t 1 , they  do  exhibit  the  phenomenon  at  any 
interior  discontinuity  of  f (x) . In  Fig.  3 . 5 we  plot  the  partial 

sums  of  the  Chebyshev  expansions  of  the  sign  function  sgn  x: 


Near  x = 0,  a Gibbs  phenomenon  is  observed;  for  fixed  x / 0, 
the  error  after  N terms  is  of  order  1/N.  In  general,  the  local 
structure  of  the  partial  sums  gR(x)  of  Chebyshev  series  near  a 
discontinuity  of  f (x)  is,  aside  from  a simple  rescaling,  given 
by  (3.14)  : 


VVirTT^1'  i<f(xo+)+f(xo->> 

pip-0 

+ |[f (x0+)-f (x0-)l  Si  (z)  (K-») 

where  |xQ|  < 1 and  z is  fixed.  This  equation  is  derived 
by  a simple  extension  of  the  argument  used  to  derive  (3.14) 
[cf.  (3.33)  below  for  the  explanation  of  the  origin  of  the 
scale  factor  l/VGJ  i- 
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Rate  of  convergence  of  Sturm-Liouville  eigenfunction  expansions 

Let  us  consider  the  expansion  of  a function  f(x)  in  terms 
of  the  eigenfunctions  <J>n  of  a Sturm-Liouville  problem:  The 
eigenfunction  $n(x)  is  a nonzero  solution  to 

p(x)glll  + (\nw(x)-q(x))  <J>n  (X)  * 0 (3.26) 

satisfying  homogeneous  boundary  conditions.  To  be  specific  in 
our  discussion  we  assume  the  boundary  conditons  0n(a)  = = 0' 

although  the  analysis  applies  more  generally.  We  assume  that 
p (x ) - 0,  w(x)  - 0,  q (x)  - 0 for  a - x - b.  We  will  also 
assume  that  the  eigenfunctions  are  normalized  so  that  they  satisfy 

b 

/ w(x)$n  (x)4>m(x)dx  = 6nm  . (3.27) 

a 

and  that  they  form  a complete  set;  the  latter  property  follows  if 

X -*•  ® as  n -*■  * (see  Courant  & Hilbert,  1953,  p.  424). 
n 

The  requirement  that  -*■  00  follows  heuristically  as  follows: 

(3.26)  suggests  that  <f>n(x)  has  a typical  spatial  scald  of  l/A^  t 
so  the  requirement  that  arbitrary  f (x)  (with  arbitrarily  small 
spatial  scale)  be  expansible  in  terms  of  { 4>n } implies  that  Xn 
must  grow  unboundedly  with  n . 
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We  wish  to  estimate  the  rate  of  convergence  of  the  eigen 
function  expansion 


f(x)  = l an$n(x) 
n=l 


(3.28) 


Using  the  orthonormality  relation  (3.27),  the  L2  - error  after 
N terms  is 


-00 

r b n 

If  I f (x)  - l an<t>n(x)|2  w (x)  dx 

V 2 

) a 

L n 

La  n=l 

p=N+l  _ 

(3.29) 


Thus,  the  L2~error  may  be  estimated  by  calculating  the  rate 

of  decrease  of  a as  n -»•  00  . 

n 

Orthonormality  of  {<)>n}  implies  that 

b 

a = / f (x)  (x)w(x)dx  . (3.30) 

n ' n 

a 

Substituting  w(x)$n(x)  from  the  Sturm-Liouville  equation  (3.26) 
gives 

b / j d<$>  \ 

an  “ T~  / I”  at  p(x)  ~&T  + q(x)0  f (x)dx  ' 

n a \ • 

Integrating  twice  by  parts,  we  obtain 


a 


n 


— P(x)  Un(x)f’  (x)-<fr*  (x)f  (x)]  | + 

A X=a 

n 


j h(x)<t>n(x)w(x)dx 
n a 


(3.31) 
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where 


h(x)  = [”  (x)  5x  + q(x)f  <x)]  /w(x). 


(3.32) 


This  integration  by  parts  is  justified  if  f is  twice  differentiable 
and  h is  square  integrable  with  respect  to  w.  Under  these  con- 
ditions and  recalling  that  <t>n  ( a ) = 4>n  ( b)  =0  , we  obtain 

an  = lP<a)$'  (a)f  (a)-p(b)*;  (b)f  (b)  ] + 0 (^-) 

An  n An 


b - b b 0 

as  n -*■  co,  since  | / h<J>  w dx  p<  / f/wdx  / T wdx  = 0(1)  as 

a a a n 

n ->  «>  . 

Nonsingular  Sturm-Liouville  problems 

To  proceed  further  we  must  distinguish  between  nonsingu- 
lar and  singular  Sturm-Liouville  problems:  a problem  is  non- 
singular if  p(x)  >0  and  w(x)  >0  throughout  a * x * b.  The 
important  conclusion  from  (3.31-32)  is  that  if  the  Sturm-Liouville 
problem  is  nonsingular  and  if  f(a)  or  f(b)  is  nonzero  then 

an  ^ ~T  tP(a)4»A(a)f  (a)-p(b)«A(b)f  (b)]  (n  - ») 


n 


(3.33) 


(Notice  that  if  <{>n(a)  = 0#  then  <{>n(x)  = 0 since  (3.26)  is 
second-order  .differential  equation  and  p(x)  ^ 0). 

It  is  well  known  [Couraht  & Hilbert  1953]  that  the  asymptotic 
behavior  of  the  eigenvalues  and  eigenfunctions  of  a nonsingular 
Sturm-Liouville  problem  are  given  by 
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Using  these  relations  in  (3.33),  we  find  that  a„  behaves  like 


This  behavior  of  a leads  to  the  Gibbs  phenomenon  in  the 

n 

expansion  (3.28)  near  those  boundary  points  at  which  f(a)  or 
f (b)  0.  The  asymptotic  behavior  (3.34-35)  implies  that  this 

Gibbs  phenomenon  is  asymptotically  identical  to  that  exhibited  by 
Fourier  sine  series  provided  we  use  the  stretched  independent  variable 


near  x - a and  a similarly  stretched  coordinate  near  x ■ b. 

If  f (a)  ■»  f (b)  ■ 0 , then  an  <<  1/n  as  n -*■  «°.  However,  a 

further  integration  by  parts  in  (3.31)  shows  that  if  the  Sturm- 

Liouville  problem  is  nonsingular  and  if  h(a)  or  h(b)  + 0, 

then  an  behaves  like  — ^ as  n «».  In  general,  unless  f (x) 

n 

satisfies  an  infinite  number  of  very  special  conditions  at  x - a 

and  x » b,  then  an  decays  algebraically  as  n-*-®. 

These  results  on  algebraic  decay  of  errors  in  expansions 

based  on  nonsingular  second-order  eigenvalue  problems  generalize 

co  higher-order  eigenvalue  problems.  For  example,  as  n+°°,  the  expansion 

00 

coefficients  in  a in  f(x)  * l a„<t>  (x),  where  U (*)  > are 

n n-0  n n n 

the  normalized  'beam'  functions 


| 


I 


behave  like  — if  f(±l)  ft  0 (implying  a Gibbs  phenomenon  at  the 

boundaries  x = ±1),  like  if  f(±l)  « 0 but  f ' (±1)  ft  0 , 

like  if  f(±l)  = f ' (±1)  = 0 but  f''"(±l)  ft  0,  and  so  on. 

n3 

Singular  Sturm-Liouville  problems 

If  p(a)  = 0 in  (3.33)  then  it  is  not  necessary  to  require  that 


f(a)  =0  to  achieve  an  << 


as  n °o  . For  this  reason, 


expansions  based  on  eigenfunctions  of  a Sturm-Liouville  problem  that 
is  singular  at  x = a do  not  normally  exhibit  the  Gibbs  phenomenon 
at  x = a.  Furthermore,  if  the  argument  that  led  to  (3.33)  can  be 
repeated  on  h(x)  given  by  (3.32)  [this  is  possible  if  p/w,  p'/w, 
and  g/w  are  bounded  and  all  derivatives  of  f are  square  integrable 
with  respect  to  w]  then  the  boundary  contribution  to  a from 
x = a is  smaller  than  -4r  as  n + 00  . If  there  are  also  no 


boundary  contributions 


from  x = b when  the  operations  leading 


to  (3.33)  are  repeated  indefinitely  [which  is  true  if  p(b)  = 0], 

then  a decreases  more  rapidly  than  any  power  of  y~  as  n •+  ® . 
n n 

The  important  conclusion  is  that  the  rate  of  convergence  of 

eigenfunction  expansions  based  on  Sturm-Liouville  problems  that  are 

singular  at  x = a and  at  x = b converge  at  a rate  governed  by 

the  smoothness  of  the  function  being  expanded  not  by  any  special 

boundary  conditions  satisfied  by  the  function. 


Fourier-Beeecl  series 

A Fourier-Bessel  series  of  order  0 is  obtained  by  choosing 
the  expansion  functions  to  be  the  eigenfunctions  of  the  singular 
Sturm-Liouville  problem 


Jxx  cRT  + Xnx*n  “ 0 (0  < x < 1) 


$n(l)  - 0,  A (0)  finite, 
n n 


(3.37) 
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Therefore,  p(x)  = w(x)  - x in  (3.26)  so  the  problem  is  singular 
at  x - 0,  but  nonsingular  at  x - 1.  The  eigenfunctions  are 


*„<*»  ' J0<Jon*1 


where  JA  is  the  Bessel  function  of  order  0 and  1 is  its 

□ 2 on 

nth  zero,  Jo^on^  “ The  eigenvalues  An  = jQn  satisfy 


?on  ^ tn-  i)if  (n-*-®)  . 


The  Fourier-Bessel  expansion  of  a function  f (x)  is  given  by 


f(x)  - l «n  J0(3onx)  . 
n*i 


The  expansion  coefficients  an  are  given  by  (3.30) 


(3.38a) 


J0<W  0 


/ tf  (t)JQ(jont)dt 


(3.38b) 


because 


I “0  (iont)2at  ' ‘ Jn'3n„>2 


0 Jon 


For  example,  the  Fourier-Bessel  expansion  of  f (x)  =1  is 


1 - - l 


n=1  W6<W 


J0(jonx) 


(3.39) 


In  Fig.  3.6  we  plot  the  10,  20,  and  40  term  partial  sums  of  the 
series  (3.39).  There  are  three  noteworthy  features  of  this 


plot: 

(i)  At  x = 1 there  is  apparently  a Gibbs  phenomenon.  In 
fact,  it  is  easy  to  show  that  this  Gibbs  phenomenon  has  the  same 


structure  as  that  for  Fourier  sine  series: 
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N 2 

' ^ ~ — 7771 — T VW  rn 
nml  Wo(W  N+! 


-)  * Si  (z) 


(N-— ) 


This  behavior  is  not  too  surprising  because  Jq  (z) *v (2/ttz)  cos(z-^-u) 
as  z-*-+®,  so- the  large  n behavior  of  (3.39)  can  be  asymptotically 

approximated  by  that  of  Fourier  series. 

(ii)  For  fixed  x satisfying  0 < x < 1,  the  error  after 
N+l  terms  of  (3.39)  is 


N , 

i * i 2- — 

nm°  3onJ0(jon) 


V^on50  " 0{S> 


(N—) 


In  fact,  the  nth  term  of  (3.39)  has  magnitude  of  order  1/n 
and  oscillates  in  sign  roughly  every  min  (jj-,  terms.  The 

error  in  such  an  oscillating  series  is  of  order  1/N  after  N terms. 

(iii)  At  x = 0,  the  series  converges  (so  there  is  no  Gibbs 
phenomenon  there)  but  the  convergence  is  very  slow  and  oscillatory. 
In  fact,  the  error  after  N terms  is  of  order  (-1)N+1//N  . 

This  follows  because 


This  slow  rate  of  convergence  near  x = 0 holds  even  though  the 
eigenvalue  problem  is  singular  at  x = 0.  There  are  two  reasons 

for  the  slow  convergence  of  Fourier-Bessel  series  near  x=0.  First, 
the  Gibbs  phenomenon  at  x = 1 affects  the  rate  of  convergence 
throughout  0 * x * 1.  In  fact,  this  is  the  sole  source  of  the 
behavior  (3.40).  Howevet,  when  f’ (x)  / 0,  slow  convergence  near 
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x gives 


"T 


I; 


1L 


x = 0 can  also  originate  because  p(x)  « w(x)  * x gives 

P'/w  = 1/x  which  is  singular  at  x - 0 so  h(x)  given  by  (3.32) 


is  singular  at 


0 if  f • (0)  + 0. 


Chebyshev  series  revisited 

Chebyshev  polynomials  are  the  eigenfunctions  of  the  singu- 
lar  Sturm-Liouville  problem  (3.26)  with  p(x)  = /l-x  , 


w (x ) - 1//1-J 


q(x)  “0,  - 1 * x * 1,  and  the  boundary  conditions 


that  * ( + 1)  be  finite.  The  eigenvalue  corresponding  to  T (x)  is 
n — n 

2 


Xn  = n‘.  since  p/w  = l-x*-  and  p'/w  = -x  are  both  finite  for 


lxl*l»  it  follows  that  the  argument  leading  from  (3.30)  to  (3.33) 
can  be  repeated  on  h(x)  given  by  (3.32)  so  long  as  f (x)  is 
sufficiently  differentiable.  Therefore,  the  Chebyshev  series 
expansion  of  an  infinitely  differentiable  function  converges 
faster  than  any  power  of  1/n  as  n -*•  »,  as  shown  fallowing  (3.23) 
by  a different  method. 

To  illustrate  the  convergence  properties  of  Chebyshev  series 
expansions,  we  study  the  rate  of  convergence  of  the  series 


” 1 

sinM~(x+a)  = 2 £ — J (Mtt)  sin  (Mna+imOT  (x) 


n=0  cn  n 


n 


Ixl  * 1 


(3.41) 

Since  Jn(M^)  -*■  0 exponentially  fast  as  n increases  beyond  Mir,  it 

follows  that  (3.41)  starts  converging  very  rapidly  when  more 

than  Mir  terms  are  included  (see  Fig.  3.7).  This  result  leads  to 

an  heuristic  rule  for  the  resolution  requirements  of  Chebyshev 
expansions.  Since  sin  Mn(x+a)  has  M complete  wavelengths  lying 
within  the  interval  |x|  < 1,  we  argue  that  Chebyshev  expansions 
converge  rapidly  when  at  least  it  polynomials  are  retained  per 
wavelength . In  general,  we  expect  that  the  Chebyshev  expansion 
of  a function  that  oscillates  over  a distance  A converges  rapidly 


if  2tt/A  polynomials  are  retained.  Fewer  polynomials  are  required 

only  (see  below) . 

if  the  reqion  of  rapid  change  of  the  function  occurs  at  the  boundary 
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error 


3 N/M 


Fig.  3.7.  A plot  of  the  Lj-error  in  the  Chebyshev  series  expansion  (3.38)  of 
sin(MTTx)  truncated  after  TN(x)  versus  N/M.  The  various  symbols  represent: 

□ M - 10;  x M ■ 20;  A M - 30;  0 M - 40.  Observe  that  the  Lj-error  approaches 

zero  rapidly  when  N/M  > it. 


The  Chebyshev  polynomial  expansion  of  a function  f(z) 
that  is  analytic  in  a region  of  the  complex-z  plane  that  includes 
the  interval  -1  < z < 1 converges  at  least  exponentially  fast 
as  n ■+  °°  . if  f(z)  has  singularities  in  the  finite-z  plane 

then 


..  , , 1/k  1 

lim  sup  |a.  | = - 

k-Mo  K R 


(3.42) 


where  R is  the  sum  of  the  semi-major  and  semi-minor  axes  of 
the  largest  ellipse  with  foci  at  z = ±1  within  which  f(z) 
has  no  singularities.  Thus,  the  L2~error  (3.29)  after  N terms  of 
the  Chebyshev  expansions  decays  to  0 roughly  like  r“N  as  N+®. 

To  prove  (3.42),  we  note  that 


n 


_2 

itc 


1 f(z)Tn(z) 


dz 


TTC 


1 f 2 -1/2 

1 I f (z)  (,1-z  ) (z 


+ /z2-l)  dz 


n 


Jc 


(3.43) 


where  C is  any  contour  that  encircles  the  interval  (-1,1) 

just  once  and  does  not  enclose  singularities  of  f(z) . 

pi — -n  /~2  -n 

Eq.  (3.43)  follows  because  2Tn(z)  = (z  + /z  -1)  + (z  - /z  -1)  , 

where  we  choose  the  branch  of  Jz^-\  satisfying  /z  -1  % z 
as  z -*■  00  . Since  (z  + v^2— 1 ) 0 as  z 00  with  this 

choice  of  branch  cut,  we  can  expand  the  contour  C to  infinity 
by  Cauchy's  theorem  and  pick  up  the  contributions  from  the 
singularities  of  f(z).  If  the  'nearest'  singularity  is  a pole 
at  z 

similarly) , then 


z with  residue  r (other  singularities  may  be  treated 
0 


n 


2i 


*3 


(z0 


+ Jz\-  1) 


-n 


(n  ■*  m) 


-58- 


T 


m 


To  complete  the  justification  of  (3.42)  we  need  only  show 
that  |zQ  + v4q-1|  ■ R . Recall  that  an  ellipse  with  foci 
at  ±1  satisfies  x2/A2  + y2/B2  = 1 with  A2  - B2  * 1 . 

If  z lies  on  this  ellipse,  then  setting  z = A cos  0 + iB  sin  0 , 

0 y 1 ■“  t 0 

it  follows  that  z.  + /z2  - 1 = (A+B)e*9  = Re*9  . 

0 0 

Let  us  give  ar  example  of  the  behavior  (3.42).  The  function 
f(z)  = tanh  (10  z)  has  poles  at  z = ± iir/20  . Thus, 

R = n/20  + A + (ir/20) 2 = 1.16934.  The  Chebyshev  expansion 
coefficients  of  f(z)  satisfy  a2^  = 0 (because  f(z)  is  an 
odd  function),  while  a^  ? 1.2679,  a3  £ - 0.4089,  a,.  i=  0.2300, 
and  so  on.  The  rms  (L2)  error  eN  [see  (3.29)] 
obtained  by  truncating  the  series  for  f(z)  after  T^fz) 
satisfies  (ea/e  ) =?  (1.175) 2 , e _/e  ^ (1.16935)2  , 

demonstrating  (3.42)  for  this  case.  The  error  eN  is  smaller 
than  0.01  for  N ^ 25,  which  again  illustrates  the  result  that 
roughly  it  polynomials  per  'wavelength'  are  required  to  resolve 
a function;  the  function  f(z)  has  a region  of  rapid  change 
near  x = 0 of  width  roughly  0.1. 

If  f(z)  is  entire,  R = 00  in  (3.42)  so  its  Chebyshev 
expansion  coefficients  decay  faster  than  exponentially.  More 
precisely,  the  method  of  steepest  descents  applied  to  (3.43)  gives 
the  following  result:  if  f(z)  is  entire  and 
f(z)  = 0 ( | z | ^ exp  \z\a)  as  z > °°  , then 

lim  sup(£n |a  | )/ (n  £n  n)  = - ^ (3.44) 

n-*-®  a 

For  example,  sin  Mir(z+a)  is  entire  with  a = 1 while  its 

Chebyshev  coefficients  in  (3.41)  satisfy  a = 0((Mtr)n/nl) 

n 

as  n ®,  in  agreement  with  (3.44).  Also,  a polynomial  has 
Chebyshev  coefficients  that  satisfy  (3.44)  with  a = 0 . 


Finally,  we  remark  that  the  Chebyshev  series  expansion 

(3.21-22)  of  an  arbitrary  function  g(x)  has  a maximum 

pointwise  error  that  does  not  differ  drastically  from  the 

smallest  possible  maximum  pointwise  error  of  any  Nth  degree 

polynomial,  the  so-called  minimax  error.  In  fact,  the  maximum 

pointwise  error  of  the  Chebyshev  series  (3.21)  truncated  after 

_ o 

T (x)  is  at  most  4 (1  + tt  £nN)  times  larger  than  the  minimax 
N 

_ 2 

error  (Rivlin  1969).  since  4(1  + tt  JlnN)  < 10  for 
N < 2,688,000,  the  Chebyshev  series  is  within  a decimal  place 
of  the  minimax  approximation  for  all  such  polynomial  approxima- 
tions. 

Legendre  series 

Legendre  polynomials  are  the  eigenfunctions  of  the  singular 

2 

Sturm-Liouville  problem  (3.26)  with  p(x)  = 1-x  , q(x)  = 0, 

w(x)  = 1 for  -1  - x - 1 and  the  boundary  conditions  are 

X = n(n+l)  and  its  eigenfunction  is  4>  (x)  - P (x) , the 
n n 11 


1-x' 


and 


Legendre  polynomial  of  degree  n.  Since  p/w 
p'/w  = -2x  are  both  finite  for  |x|  - 1,  it  follows  that  the 
Legendre  series  expansion  of  infinitely  differentiable  functions 
converges  faster  than  algebraically. 

To  illustrate  the  convergence  properties  of  Legendre  series, 
we  study  the  convergence  of  the  series 

1 

sin  Mit  (x  +a)  = [ (2n+l)  J , i (Mn)  sin  (Mia+ JniT ) P (x) 

/5m  n=0  n+*  n 

(3.45) 

Since  the  expansion  coefficients  in  (3.45)  vanish  rapidly  as  n 
increases  beyond  Mtt,  we  conclude  that  Legendre  polynomial  expansions 
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of  smooth  functions  converge  rapidly  provided  that  at  least 
tt  polynomials  are  retained  per  wavelength,  (see  Fig.  3.8)  . 

When  a discontinuous  function  is  expanded  in  Legendre  series, 
the  rate  of  convergence  is  no  longer  faster  than  algebraic.  In 
the  neighborhood  of  a discontinuity,  a Gibbs  phenomenon  occurs 
whose  local  structure  is  the  same  as  that  for  Fourier  series 
with  a suitable  stretching  of  the  coordinate.  For  example,  the 
Legendre  series  expansion  of  the  sign  function  sgn  x is 


sgn  x 


00 


= l 

n=0 


(~l)n  (4n+3) (2n) i 
22n+1 (n+1) !n!  2n+1 


(3.46) 


The  partial  sums  of  this  series  are  plotted  in  Fig.  3.9.  Three 
features  are  noteworthy: 

(i)  The  Gibbs  phenomenon  near  x = 0 has  the  same  structure 
as  that  for  Fourier  series. 

(ii)  The  error  after  N terms  behaves  like  1/N  for  |x|<l, 
x ^ o.  This  follows  from  the  fact  that  the  (2n+l)st  Legendre 
coefficient  in  (3.46)  satisfies 


a 


n 


i i\n  (4n+3) (2n) 1 „ Q,  1.x 

( * 22n+1  (n+1)  Ini  Vn- 


(n-*-®)  (3.47) 


and  the  estimate 

(x)  = 0 (— ) (n  ■+■  •) 

n /n 

for  | x | <1;  the  series  (3.46)  is  an  alternating  series  if  x 

is  fixed  away  from  zero  so  the  error  after  N terms  is  at  most 

1 2 

of  order  a P =0  (-— -) 
n n /n 
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2 3 N/M  4 


Fig.  3.8.  A plot  of  the  I^-error  in  the  Legendre  series  expansion  (3.39)  of 
sin(MiTx)  truncated  after  PN(x)  versus  N/M.  The  various  symbols  represent: 

□ M-  10;  * M - 20;  AM-  30;  0M  - 40.  Observe  that  the  L2-error  approaches 

zero  rapidly  when  N/M  > it. 


X 


This 


(iii)  The  series  converges  only  like  1//H  at  x = ±1, 

n 


follows  from  (3.47)  because  Pn(±l)  = (±1) 


for  all  n.  Thus, 
an  interior  Gibbs  phenomenon  in  a Legendre  series  expansion  has 
a 'long-range'  effect  in  the  sense  that  it  seriously  affects  the 
rate  of  convergence  at  the  endpoints  x = ±1  of  the  interval. 

In  contrast,  the  error  of  the  Chebyshev  expansion  of  sgn  x 
plotted  in  Fig.  3.5  decay  like  1/N  at  x = ±1  . This  behavior 
is  quite  general;  the  boundary  errors  of  Legendre  polynomial 
expansions  decay  to  zero  roughly  a factor  /N  slower  than  the 
boundary  errors  of  Chebyshev  expansions. 

The  rate  of  convergence  of  Legendre  expansions  of  a general 
function  f (x)  may  be  estimated  as  for  Chebyshev  expansions. 

In  particular,  the  results  (3.42)  and  (3.44)  hold  provided  that 
f (x)  satisfies  the  stated  conditions  and  (3.23)  holds  with 
only  minor  modifications. 


Resolution  of  thin  boundary  layers 

Legendre  and  Chebyshev  polynomial  expansions  give  an 
exceedingly  good  representation  of  functions  that  undergo  rapid 
changes  in  narrow  boundary  layers.  Consider  the  sequence  of 
functions  gfi  (x)  =f(x)  exp T (x-1) /<$ ] as  6+0  with  Re  S>0  for 

a fixed  smooth  function  f (x)  . As  6-*-0,  g.  (x)  develops  a 

<5 

boundary  layer  of  width  6 near  x=l.  It  may  easily  be  shown 

that  the  Chebyshev  expansion  coefficients  of  g (x)  satisfy 

o 
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, (26/ir) 


(3.48) 


1/2 


‘n 


f (l)e 


1 

'7 


n2{ 


(n-*-®  ; fin  *0(1)) 


provided  that  Re  6>0.  Thus,  if  N polynomials  are  retained, 
the  rms  error  e in  the  Chebyshev  expansion  of  (x)  satisfies 

*n  e j (Refi)N2  (N-*<*>)  . (3.49) 

The  result  (3.49)  implies  that  as  6-*-0,  the  number  of 
polynomials  required  to  reach  a specified  error  bound  increases 
only  as  l//fi,  in  contrast  to  a uniform  grid  representation  of 
g^ (x)  that  would  require  order  l/fi  grid  points  in  the  interval 
|x|<l.  In  fact,  to  achieve  1%  maximum  pointwise  error  in  boundary 
layers  of  thickness  fi  at  the  ends  of  the  interval  -1<x<1,  it 
is  necessary  to  retain  only 

N - 3//ReT"  (3.50) 


polynomials  as  fi-*-0. 


Heuristically , the  reason  that  Chebyshev  expansions  represent 

boundary  layers  so  well  is  that  the  extrema  of  Tn(x)  occur 

at  x.-cosnj/n  for  j-0,l,...,n.  Since  x0-x1~ir/2n  and 

v -x  n2/2n2  as  n-*’°°  , it  follows  that  these  polynomials  can 
n-1  n ' _2 

resolve  changes  over  distances  of  order  n 
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The  convergence  properties  of  Legendre  polynomial  expansions 
of  boundary- layer  functions  are  similar  to  those  of  Chebyshev 
expansions.  In  particular,  (3.49)  and  (3.50)  are  both  still 
valid.  In  Fig.  3.10  we  compare  the  spatial  distribution  of  the 
errors  in  Chebyshev  and  Legendre  polynomial  expansions  of  the 
function  g(x)  = e100(x-l)^  which  has  a narrow  boundary  layer 
of  width  1/100  near  x=l.  Apparently  for  x away  from  the 
boundaries  x=±l,  the  Legendre  expansion  has  somewhat  smaller 
errors,  while  near  x=±l  the  Chebyshev  expansion  has  smaller 


errors . 


The  Legendre  expansion  gives  the  polynomial  Q (x)  of 

N 

degree  N that  minimizes 


/ lg(x)  - Qn(x) I dx 


while  the  Chebyshev  expansion  gives  that  Q„  that  minimizes 

N 


/ | g (x)  - Q (x) | 2 (l-x2) _1/2  dx. 

-1 


The  Chebyshev  expansion  also  gives  a smaller  maximum  error 

max  | g (x)  - Q (x)  | 

| x | < 1 N 

than  the  Legendre  expansion  by  roughly  a factor  2//N  ; 
as  remarked  above,  the  Chebyshev  QN(x)  is  usually  remarkably 
close  to  the  minimax  polynomial  that  minimizes  the  maximum 
error. 
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Laguerre  polynomials  are  the  eigenfunctions  of  (3.26)  with 

p(x)  = xe-x,  q(x)  = 0,  w(x)  = e~x  for  0 < x < » with 
1 x 

- x X 

e * <)>n(x)  bounded  at  x = 0 and  <*>.  The  n eigenvalue  is 

X = n and  the  associated  eigenfunction  is  <f>  (x)  = L (x)  , 
n n n 

the  Laguerre  polynomial  of  degree  n.  If  f (x)  and  all  its 
derivatives  are  smooth  and  satisfy 


f (x)  = 0(eaX) 


(x  °°) 


for  some  a < £ , it  is  easy  to  show  by  retracing  the  derivation 
of  (3.33)  from  (3.30)  that  the  Legendre  expansion 


f(x)  = l anLn(x) 
n=0 

converges  faster  than  algebraically  as  the  number  of  terms  N -*•  ®. 

9 

To  illustrate  the  rate  of  convergence  of  Laguerre  series, 
we  consider  the  expansion  of  sin  x : 

sinx  = J0  ~udrr/2  cos|j(n+l)]Ln(x)  (3.51) 


which  converges  for  all  x,  0 - x < °°.  Since 


L (x)  ^ — e^xx-^n-^  cos  (2/nx -Jit  ] , 

n /? 


[see  Erdelyi  et  al  1953,  Vol . II,  pg.  200)  it  follows  that  if 
N > > x,  then  the  error  after  N terms  at  x is  roughly 


This  error  is  small  only  if  N In  2 > x or  N~  1.44x.  Since 
the  wavelength  of  sin  x is  2tt  , Laguerre  expansions  require 
approximately  9.06  polynomials  per  wavelength  to  achieve  high 
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accuracy.  (This  figure  may  be  reduced  to  about  6.53  polynomials 
per  wavelength  by  using  the  modified  Laguerre  expansion 
\ anLn(x)e”ax  and  optimizing  the  choice  of  a.)  Thus,  Laguerre 
expansions  require  many  more  terms  to  resolve  a function  of  given 
complexity  than  do  either  Chebyshev  or  Legendre  expansions.  The 
reason  is  that  significant  weight  is  given  to  x -t  + 00  in  the 
Laguerre  series  where  sin x has  an  essential  singularity. 

Iii  Figs.  3.11-13,  we  plot  the  partial  sums  of  (3.51)  with 
N = 10,  20,  and  40  terms.  Observe  that  the  number  of  wavelengths 
of  sin x represented  accurately  by  (3.51)  is  roughly  N/9. 

Hermite  expansions 

Hermite  polynomials  satisfy  (3.26)  with  p = e , q(x)  = 0, 

2 _ix2 

w(x)  = e”x  for  - oo  < x < 00 , <J>n(x)e  5 bounded  as  |x|  ■*■<*>. 

The  Hermite  polynomial  Hn(x)  of  degree  n is  associated  with 
the  eigenvalue  = 2n.  If  f (x)  and  all  its  derivatives  satisfy 

2 

f (x)  = 0 (eax  ) ( | x | -*  «,) 

for  some  a < i,  then  the  Hermite  expansion 


f(x)=l  anHn<x) 

n=0 

converges  faster  than  algebraically  as  the  number  of  terms  N 
This  is  proved  by  retracing  the  steps  leading  from  (3.30)  to  (3.33). 

To  study  the  rate  of  convergence  of  Hermite  series,  we  consider 
the  expansion  of  sin  x : 


sin  x 


l 

n=0 


1 

22n+l (2n+l)  1 


H2n+1 (x) 
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(3.52) 


J 


Since  the  asymptotic  behavior  of  HR(x)  is  given  by  [Erdelyi, 
et.  al  1953,  vol.  II,  pg.  201] 

Hn(x)  -v  e^x  cos  x “ *nir) 


as  n -*•  00  for  x fixed,  it  follows  that  the  error  after  N 
terms  of  (3.52)  goes  to  zero  rapidly  at  x only  if  N ~ Xog"x 
This  result  is  very  bad;  to  resolve  M wavelengths  of  sin x 
requires  nearly  M2  Hermite  polynomials!  [By  expanding  in  the 


series  \ an^n^x^e  ^ an<*  the  choice  of  a,  it  is 

possible  to  reduce  the  number  of  required  Hermite  polynomials  to 
5 

about  ^ u =7.85  per  wavelength,  but  this  is  still  quite  poor.] 
Because  of  the  poor  resolution  properties  of  Laguerre  and 
Hermite  polynomials  the  authors  doubt  they  will  be  of  much  prac- 
tical value  in  applications  of  spectral  methods. 


4 . Review  of  Convergence  Theory 


The  fundamental  problem  of  the  numerical  analysis  of 
initial  value  problems  is  to  find  conditions  under  which 
uN(x,t)  converges  to  u(x,t)  as  N for  some  time  in- 

terval 0 £ t £ T and  to  estimate  the  error  ||u  — uN  1 1 . 

The  principal  result  is  the  Lax-Richtmyer  equivalence  theorem 
which  states  that  stability  is  equivalent  to  convergence  for 
consistent  approximations  to  well-posed  linear  problems.  The 
terms  stable,  convergent,  and  consistent  relate  to  technical 
properties  of  the  approximation  scheme  which  are  defined  below. 

An  approximation  scheme  (2.5-6)  is  stable  if 


L„t 

lie  N ||  < K(t> 


(4.1) 


for  all  N where  K(t)  is  a finite  function  of  t . Here 
the  operator  norm  is  defined  by 


lie 


V 


II 


max 

ue# 


LNfc 

lie  N u || 

iiuii 


An  approximation  scheme  is  convergent  if 

||u(t)  - uN(t)  II  -*-0  as  N -*■  ® 

for  all  t in  the  interval  0 £ t £ T and  all  u(0)eX  and 
f(t)  e)/.  Finally,  an  approximation  scheme  is  consistent  if 


-►  0 


II  Lu  - Lnu|| 

llu  - pN«ii 


0 
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(4.2) 





as  N «*  for  all  u in  a dense  subspace  of  . 

The  classical  Lax-Richtmyer  equivalence  theorem  relating 
the  above  definition  states  that  "a  consistent  approximation  to 
a well-posed  linear  problem  is  stable  if  and  only  if  it  is 
convergent."  In  this  monograph  we  are  confronted  with  some 
subtleties  regarding  the  notions  of  stability  and  convergence. 
Because  a precise  understanding  of  the  ideas  of  stability  and 
convergence  is  important  to  the  theory  of  algebraic  stability 
given  in  Sec.  5,  we  outline  here  the  proof  of  the  equivalence 
theorem. 

Proof  of  the  Equivalence  Theorem 

To  show  that  stability  implies  convergence  we  use  (2.1)  and 
(2.5)  to  obtain 


3t 


VU'V 


+ Lu  - L U + 
N 


Thus, 

LNt 

u(t)  - lyt)  = e N [u(0)-uN(0)J 


t L ( t-  s ) 

+ / e N [Lu  ( s)  -LjjU  (s)  +f  (s)  “f  jj  (s)  ] ds  . (4.3) 

0 

Using  (4.1)  and  (4.3)  and  the  triangle  inequality  we  obtain  the 
estimate 


||u(t)-UN(t)  ||  < K(t)  ||u(0)rUN(0)  || 


K(t-s)  ( ( | Lu ( s ) -LNu(s)  II 


+ 


||f(s)-fN(s)  III  ds 
(4.4) 


Thus,  if  u(t)  belongs  to  the  dense  subspace  of  satisfying 

(4.2)  and  if  f(t)  belongs  to  the  dense  subspace  of  satisfy 

ing  ||f  - PNf  ||  -*-0  as  N , then  ||n(t)  - uN(t)  ||  - 0 
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as  N -*  • . Since  all  solutions  u(t)  of  (2.1)  can  be 

approximated  arbitrarily  well  by  functions  satisfying  (4.2), 

the  proof  that  stability  implies  convergence  is  completed. 

Conversely,  to  show  that  convergence  implies  stability, 

L„t 

we  first  observe  that,  for  any  ueH,  )|e  u||  is  bounded 

for  all  N and  each  fixed  t . In  fact,  convergence  implies 


0 < 


LNt 

lie  N u||  - 


l|e“u|| 


LNt 

< ||e  N u 


Lt  .. 
" e u || 


(N  “>) 


while  well-posedness  requires  that  ||eLtu||  is  finite.  How- 
L„t 

ever,  max||e  u||  may  depend  on  u and  on  t , so  stability 
N 

is  not  yet  proved.  To  complete  the  proof  we  use  the  fact  that 

^ is  a Hilbert  space.  The  principle  of  uniform  boundedness 

L t 

(Richtmyer  & Morton  1967)  implies  that  if  ||e  u||  is  bounded 

Lj.t 

as  N -*■  «>  for  each  t and  ueW'  then  ||e  ||  is  bounded  as 

N -+■  » for  each  t . This  proves  stability  and  completes  the 
proof  of  the  equivalence  theorem. 

Using  the  equivalence  theorem,  the  study  of  the  convergence 
of  discrete  approximations  to  the  solutions  of  initial-value  problems 
is  reduced  to  the  study  of  the  stability  of  the  discrete  approxima- 
tions, assuming  the  approximations  are  consistent.  Thus,  the  de- 
velopment of  conditions  for  the  stability  of  families  of  finite- 
dimensional operators  is  of  primary  interest  in  numerical 

analysis . 

Von  Neumann  Stability  Condition 

The  simplest  condition  for  stability  is  due  to  von  Neumann. 

Let  us  suppose  that  the  Hilbert  space  possesses  the  inner  product 

( , ) . Using  the  inner  product,  we  define  (neglecting  the  compli- 

* 

cations  due  to  boundary  conditions)  the  adjoint  L of  an  operator 

L as  that  linear  operator  that  satisfies 
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(u, Lv) 


(L  ufv)  for  all,  u,v  in  ^ . For  the  finite  dimen- 


m 

sional  approximation  1^,  the  matrix  representation  of  1^  is 
the  adjoint  of  the  matrix  representation  of  LN  (see  Sec.  2) . 

The  operator  is  said  to  be  a normal  operator  if  commutes 

with  to  LnL^  - L^Ln  . 

The  von  Neumann  stability  condition  is  that  stability  of 


normal  operators  is  equivalent  to  the  condition 


Re  AN  < C (4. 

where  is  any  of  the  eigenvalues  of  any  of  the  operators 

N 

L.,  and  C is  a finite  constant  independent  of  N . To  prove 
N 

* 

this,  we  note  that  if  is  normal,  then  LN  and  LN  as 

well  as  exp(LNt)  and  exp(LNt)  are  simultaneously  diag- 


(4.7) 


nolizable.  Therefore, 


L„t  J 

lie  N ||2 


(u,eL'‘teVu)  2(ReXN)t 

“ <“>“>  ' 7*  e 


where  XN  are  the  eigenvalues  of  LN  . Thus,  the  von  Neumann 
condition  (4.7)  is  equivalent  to  the  stability  definition  (4.1) 
with  K (t ) = exp(2Ct)  . 

The  von  Neumann  condition  gives  an  operational  technique 
for  checking  stability  of  normal  approximations:  compute  the 
eigenvalues  of  LN  and  check  that  the  real  parts  of  the  eigen- 
values are  bounded  from  above. 

Example  4.1:  Symmetric  hyperbolic  system  with  periodic 
boundary  conditions 

Let  us  apply  the  theory  just  discussed  to  the  stability 
of  difference  approximations  to  the  m-component  symmetric 

hyperbolic  system 


3u(x,t) 

3t 


au(x,t) 


(4.8) 


with  periodic  boundary  conditions  iS(0,t)  - u(l,t)  . 
Here  u is  an  m-component  vector  and  A is  a symmetric 


m x m matrix. 


If  we  discretize  in  space  using  second-order  centered 
differences,  we  obtain 


Zi  . AVi_lVi 

at  2Ax 


(j  = 1,2, ...,N) 


(4.9) 


U0(t)  = UN(t)  ' Ul(t)  = Vl(t) 

where  uk(t)  = u(k/N,t)  and  Ax  = 1/N  . The  system  (4.9) 
is  equivalent  to  the  system  of  mN  equations 


3u  _ ~ 

= B U 


(4.10a) 


where  u is  the  column  vector  whose  transpose  is 

uT  = <yu2,...^).  Here  B is  the  mN  x mN  matrix  given 

by  the  Kronecker  product 


B = A 0 D , 


(4.10b) 


where  A is  the  m x m matrix  in  (4.8)  and  D is  the 
N x n matrix 


0 10  0 
-10  10 


. 0 -1 


0-1  0 1 ...  0 0 


0 0 0 0 
10  0 0 


0 1 


D is  anti-symmetric  (so  D = -D  and,  hence,  D is 
normal)  so  it  has  eigenvalues  that  are  either  0 or  pure 
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i sin(2itkAx)/Ax  for  k ■ 0,1,..., N-l.  Thus,  the  norm 
of  exp(Bt)  satisfies 

||exp(Bt)  ||  - max  ||exp(iA  sin(2nkAx)  t/Ax  ||  * 1 , 

0<k<N 

where  we  use  the  fact  that  A is  symmetric  so  it  has  real 
eigenvalues. 

Kreiss  Matrix  Theorem 

If  the  approximate  evolution  operatois  1^  are  not  normal, 
conditions  guaranteeing  stability  are  much  harder  to  obtain. 

The  von  Neumann  condition  (4.7)  is  still  necessary  for 

stability  (why?),  but  it  is  not  sufficient  to  ensure  stability. 

One  important  case  in  which  stability  conditions  can  be  found 

is  for  the  problem  studied  in  Example  4.1  with  A no  longer 

symmetric.  The  appropriate  generalisation  is  to  assume  that  the 

approximation  Lx,  has  the  form  L„  - A & D%,  where  A is  a fixed 
N N N 

m x m matrix  (possibly  not  normal)  and  D..  is  an  N-dimensional 

N 

normal  matrix.  It  is  easy  to  show  that 

||  exp(L^t)  ||  - max  ||exp(XNAt)  ||  (4.11) 

% 

where  AN  is  any  of  the  eigenvalues  of  DN  • A stability 
condition  for  (4.11)  will  be  obtained  below.  To  do  this,  we 
generalize  (4.11)  and  seek  conditions  for  the  stability  of  a family 
of  m x m matrices  A(to)  , where  to  is  an  arbitrary  parameter. 
That  is,  we  seek  conditions  such  that 

max  ||expfA(w)  tj  ||  <_  K(t)  , 
to 

where  K(t)  is  a finite  function  of  t.  Once  these  general 
ndttions  are  found,  they  can  be  specialized  to  give  stability 
"rviitions  for  families  of  the  form  exp(LNt)  where  LN«A^D^  with 


Dn  normal  by  simply  choosing  A (w)  ■ Aw  where  <o  is  any  of 
the  eigenvalues  of  any  of  the  matrices  . 

The  basic  result  on  the  stability  of  families  of  m x m 
matrices  is  the  Kreiss  matrix  theorem  (Kreiss  1962) : 

For  any  family  A(io)  of  m x m matrices,  each  of 

the  following  statements  implies  the  next: 

(i)  There  exist  symmetric  matrices  H(w)  satisfying 

H (to)  A(io)  + A*(<o)K(io)  < 0 and 

I < H (to)  , ||  H (to)  ||  £ C for  some  constant  C . 

(ii)  ||exp  (A(<o)  t]  ||  < C for  all  t >.  0 . 

( iii)  (Re  A ) ||  ( XI-A (to) -1  ||  < C'  for  some  constant  C' 

and  all  A satisfying  Re  A > 0 . 

(iv)  There  exist  matrices  H(u>)  satisfying  (i)  with 

||  H (to)  ||  < K(m)C'  where  C'  is  the  constant 
appearing  in  (iii)  and  K(m)  depends  only  on 
m and  not  only  the  family  A (to)  . 

Observe  that  for  a family  of  matrices  A(w)  to  satisfy 
the  conditions  of  this  theorem  it  is  necessary  that  all  the 
eigenvalues  of  all  the  matrices  have  non-positive  real  parts. 
Otherwise  there  would  be  some  to  and  some  eigenvector  u satis- 
fying ||  exp  (A  (to)  t]  u ||  -*■  «>  as  t -*■  <»  violating  (ii). 

The  most  important  relation  implied  by  this  theorem  is  the 
implication  that  (iii)  implies  (ii)  with  C —■  K(m)C'  That  is, 
for  any  m x m matrix  A all  of  whose  eigenvalues  have  nonposi- 


tive real  parts 


(4.12) 


exp(At)  ||  £ K'  (m)  max  (Re  A)  ||  (AI-A) 

Re  A > 0 


where  K' (m)  is  a finite  function  of  m . 

An  elementary  proof  of  (4.12)  has  recently  been  given 

by  Laptev  (1975)  and  improved  by  C.  McCarthy  (private  communica- 
tion to  G.  Strang,  1975).  Lapfev  observes  that  if  v > 0 , then 
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i 

7 


i 

^ L. 


V+i“ 


eXt(AI-A)-1  dA 


J°°  eiyt  (v  + iy-A)-1  dy  , (4.13) 


y_jco 


as  may  be  proved  by  shifting  contours  in  the  complex  plane. 

Since  each  entry  of  (v  + iy-A)  1 is  a rational  function  in  y 
of  degree  at  most  m , the  derivatives  of  the  real  and  imaginary 
parts  of  each  entry  can  change  sign  at  most  4 m times  when  y 
increases  from  to  °°  . On  any  y-interval,  say  a £ y b , 

where  the  real  and  imaginary  parts  of  an  entry  in  (v+iy-A)  1 
are  monotonic,  the  second  mean-value  theorem  implies 


fb 

cos  yt  f(y)  dy  = f(a) 
Ja 


iin(ct)  - 


- sin(at)j 


+ f (b)  |sin(bt)  - sin(ct)j 


< - max | f (y)  | , 

z y 


for  some  c satisfying  a < c < b where  f(y)  is  the  real  or 
imaginary  part  of  an  entry  in  the  matrix  (v+iy-A)-1.  If  we  apply 
this  kind  of  inequality  to  the  right  side  of  (4.11),  it  follows 
that  for  all  i,j 

< llJL  max  (v+iy-A)  71  . (4.14) 

t y ID 

If  it  is  true  that  the  matrix  norm  has  the  property  that 
!®ij|  < 0^^  for  all  i,j  implies  ||b||<.||c||  , then  (4.14) 
impl ies 

elyt(v  + iy-A)  1 dy  < max  (v  + iy-A)-1  (4.15) 

c y 

Choosing  v ■ 1/t  in  (4.13-15)  gives  (4.12)  with  K'(m)  = 64  m . 


,oo 

J — 00 


iyt  (v  + iy-A)  T1 
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There  are  three  important  matrix  norms  in  which 

| Bi j | <.  for  all  i,j  implies  ||B||  <.  ||c||  , namely 

the  matrix  norms  induced  by  the  L.  , , and  L vector 

l z 00 

norms.  This  is  shown  using  the  relations 


II B ||  x 

l|B|l  2 

1 1 B 1 1 oo 


m 

max  E I B . . | 

j i-i  « ' 


sup 


II X II  2 = 1 i=1  3=1 

lly||2=i 


m m 
E E B . .x  .y. 


' • J A ' J "I 

lj  3 


m 

max  E | B . . | 
i j=l  ^ * 


which  hold  for  all  matrices  B . In  other  norms  lBijl  — Cij 
may  not  imply  ||b||  < ||c||  but  the  equivalence  of  all  matrix  norms 
implies  ||  B ||  < F(m)  ||C||  for  some  finite  function  of  the 
dimension  m.  Thus,  (4.12)  is  obtained  with  K'(m)  = 64 m F (m)  . 

The  functions  K(m)  appearing  in  statement  (iv)  of  the 
Kreiss  theorem  and  K' (m)  appearing  in  (4.12)  need  not  be  equal. 

It  follows  from  the  Kreiss  theorem  that  K'(m)  < K(m)  . Kreiss 
showed  only  that  K(m)  = 0{mm)  as  m -*■  ® ; this  is  much  too 
conservative.  Miller  & Strang  (1965)  showed  that  K(m)  = 0 ( Cm) 
as  m ■+  ® for  some  constant  C > 1 . 

In  the  case  of  a normal  family  of  matrices  A(u>)  the  con- 
ditions of  the  Kreiss  matrix  theorem  are  trivially  satisfied: 
if  the  eigenvalues  of  A(u>)  have  negative  real  parts  then 
||explA(u)  t]  ||  <.  1 for  all  t > 0 and  w . 
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Non-Normal  Approximations 


The  Kreiss  matrix  theorem  applies  to  approximations  of  the 
form  Ln  * A^Dn,  where  A is  a fixed  dimensional  non-normal 
matrix  and  DN  is  an  N-dimensional  normal  matrix.  This  type 


of  operator 


Ln  is  commonly  encountered  in  the  solution 


of  initial-value  problems  with  periodic  boundary  conditions. 

On  the  other  hand,  non-periodic  boundary  conditions  usually  lead 
to  problems  in  which  the  non-normality  affects  the  N-dependent 
operator  DN.  When  finite-difference  methods  are  used  for  such 
problems,  the  deviation  of  DN  from  a normal  operator  is  frequently 
'small* 


Example  4.2:  Non-normality  of  a difference  approximation  to  a mixed 
initial-boundary  value  problem 
A difference  approximation  to  the  mixed  initial-boundary 
value  problem 


$ + - f(x,t)  (0A1,  t>0), 

u(Oft)  = 0; 


u (x, 0)  * g(x) 


is  given  by 


3u-i  u-i+l"UT-l 

-gJ-  + -3+^-3—  = f (jh,t)  (l±j±N) 


(4.16) 


where  Uj  (t)  = u(jh,t)  anr1.  we  set  UQ(t)  -0  and  u^^tt)  “ 2uN(t) 
-u„  , (t) . The  latter  condition  is  an  extrapolation  condition 
which  ensures  that  (4.16)  is  a closed  system  of  equations.  This 
approximation  has  the  matrix  representation 
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0 0 
0 0 


0 

0 


,0  0 0 0 

\ 0 0 0 0 


The  departure  of  from  a normal  matrix  is  a matrix  of 

rank  1 in  the  lower  right-hand  corner.  For  problems  of  this 
kind,  extensions  of  von  Neumann  stability  analysis,  like  that 
introduced  by  Godunov  & Ryabenkii  (196  3)  and  extended  by 
Kreiss  (see  Kreiss  & Oliger  1973) , apply. 

Unfortunately,  the  class  of  semi -discrete  approximations 
investigated  in  this  monograph  include  problems  that  cannot  be 
easily  analyzed  either  by  straightforward  von  Neumann  stability 
analysis  or  by  the  Godunov- Ryabenkii  or  Kreiss  analysis. 

In  contrast  to  the  classical  problems  of  the  numerical  analysis 
of  difference  methods  for  initial-value  problems,  spectral 
approximations  are  frequently  not  even  approximately  normal . 
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5.  Algebraic  Stability 

In  this  section,  we  develop  a theory  of  stability  and 
convergence  which  generalizes  the  classical  theory  discussed 
in  Sec.  4.  As  will  be  shown  by  examples  in  Sects.  6-8>  this 
generalized  stability  theory  is  well  suited  to  study  the  con- 
vergence of  spectral  methods. 

A spectral  approximation 


LNUN  + fN 


(5.1) 


to  the  initial-value  problem  ufc  = Lu  + f is  called 
algebraically  stable  as  N -*■  » if 


|| e N ||  £ NrNstK(t) 


(5.2) 


for  all  sufficiently  large  N , where  r,  s,  and  K(t) 
are  finite  for  0 i t i T . 

It  may  at  first  seem  that  the  Lax-Richtmyer  theorem  shows 
that  algebraically  stable  approximations  cannot  be  convergent 
unless  (5.2)  holds  with  r < 0,  s <.  0 . In  fact,  if  we 
demand  that  the  approximations  converge  for  all  u(0)  and 
f (t)  in  the  Hilbert  space  H , this  conclusion  is  correct. 
However,  it  is  possible  for  approximations  that  satisfy  (5.2) 
with  r > 0 or  s > 0 to  converge  on  a dense  subset  of 
the  Hilbert  space  in  which  the  only  functions  for  which  con- 
vergence is  not  obtained  are  highly  pathological.  In  fact,  if 
p * r ♦ sT  > 0 but  p is  smaller  than  the  order  of  the 
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spatial  truncation  error  of  a particular  solution  u(x,t) 


# 


i.e. 


. ] 


NP  ||  Lu  (t) 

- LNu(t)  ||-^0 

(N  -+•  <*>) 

(5.3a) 

NP  ||  u (0)  - 

VO)  ||  - 0 

(N  •+•  00 ) 

(5.3b) 

NP  ||  f (t)  - 

*M(t)||  - o 

(N  -*•  oo) 

(5.3c) 

for  all  0 < t <.  T , then  (4.4)  and  (5.2)  imply  that 

||  u ( t ) - ^(t)  ||  -*•  0 (N  -*■  °°) 

for  0 <.  t <.  T . Thus,  algebraic  stability  implies  con- 
vergence in  that  subspace  of  X satisfying  the  conditions 
(5.3).  If  this  latter  subspace  is  large  enough,  an  algebraic- 
ally stable  method  can  still  be  very  useful  although  it  cannot 
yield  convergent  results  for  all  initial  conditions  u(0)  and 
forces  f(t)  . Since  spectral  methods  are  normally  infinite- 
order  accurate,  algebraic  stability  implies  convergence  for 
such  spectral  methods. 

In  the  examples  of  algebraic  stability  given  in  Sects.  7-9, 
we  find  r <.  j , s <.  0 , and  K(t)  < M . In  this  case, 
algebraic  stability  implies  convergence  so  long  as  (5.3)  holds 
with  p < j . Thus,  the  approximation  need  not  be  infinite- 
order  accurate  to  achieve  convergence.  However,  we  develop  the 
general  theory  of  algebraic  stability  here  in  the  expectation 
that  it  will  find  application  to  spectral  methods  for  high-order 
equations  in  which  p may  be  large. 


L 
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Our  definition  of  algebraic  stability  is  very  similar 
to  the  notion  of  s-stability  introduced  by  Strang  (1960)  . 
However,  our  motivation  is  slightly  different.  Strang  intro- 
duced s-stability  to  study  the  convergence  of  time-discretized 
initial-value  problems  in  which  the  norm  of  the  evolution 
operator  grows  as  a power  of  the  time  step.  We  shall  return 
to  this  concept  when  we  discuss  generalized  stability  in  Sec.  9. 

Let  us  give  an  illustration  of  the  need  for  a theory  of 
algebraic  stability.  In  Sec.  8,  we  will  discuss  Chebyshav 
polynomial  spectral  methods  to  solve  the  one-dimens  .nal 
wave  equation  ut  + ux  = with  boundary  conditions 

u(-l,t)  = 0 . Unfortunately  this  problem  is  not  well  posed 
in  the  Chebyshev  norm 


iiuii 


2 


,1 

•-1 


u2  (x) 


dx  • 


In  fact,  if 

u (x, 0) 


if  { x 1 < e 
if  | x | > e f 


then  the  solution  of  u.  + u = 0,  u(-l , t ) = 0 at  t = 1 is  given 

X 

by 

11  - | + £ 1-e  < x < 1 

0 x < 1-e 
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- MM 


||u  (x,  0 ) 1 1 2 ^ e (e  -*■  0+) 


1 1 u ( x jf  1 ) 1 1 2 ^ | fll  (e  -*■  0+)  , 


(e  -*•  0+) 


(5.4) 


In  fact,  ||eLt||  = « for  0 < t < 2 , ||eLt||  = 0 

for  t > 2 , so  the  one-dimensional  wave  equation  is  not 
well  posed  in  the  Chebyshev  norm. 

Since  the  finite-dimensional  approximations  LN  to  L 
given  by  Galerkin,  tau,  and  collocation  approximation  (see 
Sec.  2)  should  converge  as  N -►  «°  , it  follows  that  we  may 
expect 

||exp(LNt)  ||  -*•  » 

as  N -*■  00  in  the  Chebyshev  norm.  To  estimate  the  rate  of 
divergence  of  ||exp(LNt)||  as  N -*■  00  we  argue  that 
Chebyshev  polynomials  of  degree  at  most  N can  resolve  dis- 
tances of  at  most  order  1/N  interior  to  (-1,1)  so  we 
may  reasonably  guess  on  the  basis  of  (5.4)  with  e = 1/N  that 


||exp(LNt)||  = 0 ^N  T J (N  -*•  ®)  . (5.5) 

This  result  is  justified  by  the  numerical  results  presented 

in  Table  8.3.  Eq.  (5.5)  implies  that  Chebyshev-spectral  approximations 
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to  the  one-dimensional  wave  equation  are  not  stable  but  are 
algebraically  stable  with  r = 1/4  and  s = 0 in  (5.2). 

Notice  that  algebraic  stability  in  one  norm  implies 
algebraic  stability  in  all  algebraically  equivalent  norms. 
Thus,  algebraic  stability  is  equivalent  in  all  of  the  Lp 
norms  1 £ P <.  00  because  these  norms  are  algebraically 
equivalent  in  N-dimensional  vector  spaces  (i.e.,  they  differ 
from  each  other  only  by  a fixed  power  of  N ) . To  show  this, 
we  recall  that  the  L norm  of  a vector  a = (a.,..., a.,) 

p X N 

is  defined  by 


If  q = pa  with  0 < a < 1 , then 


by  Holder's  inequality.  Therefore,  for  all  p > 1 t 

5”1 

NP  1 1 a ||  x < 1 1 a 1 1 p 
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The  verification  of  algebraic  stability  for  spectral 
methods  leads  to  a general  problem  in  matrix  theory.  Suppose 
that  A.,  (N=l,  2, . . . , ) is  a one  parameter  family  of  matrices. 
We  will  find  conditions  on  the  members  of  the  family  such  that 
exp  (Ajjt)  is  algebraically  stable.  We  will  use  only  the  L2 
norm  since  the  others  are  equivalent  to  it. 

Conditions  for  Algebraic  Stability 

Let  {an>  be  a family  of  N x N matrices  where 
||  A^  1 1 = 0(Na)  (N  -*•  °°)  for  some  finite  a . A necessary 

and  sufficient  condition  for  algebraic  stability 

|| e N ||  = 0 ^NrNSt^  (N  -*•  ») 

is  that  there  exist  a family  {Hn>  of  Hermitian  positive- 
definite  matrices  such  that 
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IlH"1  II  ||HN||  =■  o (Nb) 


HA*V»  i °<N>HN  , 


c (N)  < d log  N 


(N  -►  ®)^  (5.7a) 


(5.7b) 


(5.7c) 


for  all  sufficiently  large  N where  b and  d are  finite 
numbers  independent  of  N . 

To  prove  sufficiency  we  use  the  Lie  formula 


(C+D) 


fc  = lim( 

n-n»  \ 


Ct/n  Dt/n' 
e e i 


(5.8) 


which  is  valid  for  arbitrary  matrices  C and  D . This 
formula  is  proved  at  the  end  of  this  section.  If  we  define 


r 1 1 1 1 I 

1 I 7 ~ 2 T * I 

" 2 LHN  ¥n  +Hn  S «N  J, 


(5.9) 


r 1 . 1 _ i n 

|>  Vn  1 - hn  7 N h/J 


and  note  that 


exp  [ANt] 


_ 1 Pi  .1  I 1 

hn  exp  j^HN  tj  Hn  , 
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fc 


it  follows  from  the  Lie  formula  that 


_ . 1 1 
. . „ ~ 1 ( Ct/n  Dt/n\n  7 

e ■ H«  (e  e j bn  • 

However,  it  follows  from  (5.7b)  that,  since  C is  a 
symmetric  matrix, 

||ect'n||  < eot/n  . 

Also,  D is  an  antisymmetric  matrix  so  that 

l|eDt/n||  - 1 . 

Therefore,  (5.10)  gives 

l|eA,<t||  < ect  ||HH'5|| 

proving  algebraic  stability. 

In  order  to  prove  that  the  conditions  (5.7)  are  also 
necessary  for  algebraic  stability  we  define 

BN  = An  ‘ (r+1)  1°9<N)1  • 


1 

I|h/||  < 


eCtNb/2 


Therefore, 


Bwt 

le  N || 


(N  °°)  . 


.10) 


-92- 


By  Liapounov's  theorem  (Barnett  & Storey  1974)  there  exists 


a Hermitian  positive-definite  matrix  H such  that 


V«  + bb  “n  - -1  ■ 


(5.11) 


Thus, 


VS*  + *N  HN  = -I  + 2<r+l)  log  N Hn  < c(N)Hn, 


where  c(N)  = 2(r+l)  log  N . In  order  to  complete  the 

proof  of  (5.7)  we  need  to  estimate  the  norms  of  HN  and 

Hn_1  . It  can  be  easily  verified  that  an  explicit  formula 

for  H.,  is 
N 


■»  - 1 


" V > 1 .. 

e e dt  . 


Therefore, 


oo  ^ oo 

||hn||  < lle^ll  J|  6Bn  tJ|  dt  < n2s  n 
n n 


"2t  dt  ^ N2s 


if  2 JtnN  > 1,  i.e.,  N >.  2 . Also  from  (5.11)  we  obtain 


Vn’1  + VV  - - <V1)2. 
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so  that 


IlHjj”1!!2  < 2 ||bn||  ||hn_1|| 

or 

||Hn_1  ||  < 2 || BN  ||  = 0(Na)  (N  - co)  (5.12) 

This  completes  the  proof  of  the  necessity  of  (5.7) . 

The  condition  for  algebraic  stability  given  in  (5.7) 

implies  that  for  every  algebraically  stable  problem,  there  is 

a new  norm  induced  by  the  Liapounov  matrices  H„  which  is 

N 

algebraically  equivalent  to  the  original  norm  and  in  which 
the  problem  is  stable  in  the  classical  sense. 

The  above  result  gives  a method  for  checking  numerically 
the  algebraic  stability  of  a family  {A^}  of  matrices  satis- 
fying ||An||  = 0(Na)  as  N -*■  00  : 

(i)  We  check  that  the  real  parts  of  the  eigenvalues 
of  An  are  bounded  from  above  by  s log  N ; 
otherwise,  the  family  of  matrices  A^  are  alge- 
braically unstable. 

(ii)  We  introduce  BN  = AN  “ (s+l)log(N)I  and 

compute  the  Liapounov  matrix  HN  such  that 
* 

HNBN  + BN  HN  = "I  • There  are  several  numeri- 
cally efficient  techniques  to  compute 
(Bartels  & Stewart  1972)  . 
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(iii)  To  verify  alqebraic  stability  the  condition  number 

of  Hu  must  be  bounded  by  Nb  for  some  finite  b 
N 

as  N + - . Noting  (5.12),  it  is  only  necessary 
to  verify  that  the  eigenvalues  of  HN  are  bounded 
from  above  by  some  finite  power  of  N as  N ® 

This  procedure  is  applied  in  Sects.  7-8  to  verify  alqebraic 
stability  of  model  problems.  Since  (5.7)  gives  a necessary 
and  sufficient  condition  for  algebraic  stability,  if  these 
conditions  do  not  hold  the  family  of  matrices  AN  is  alge- 
braically unstable. 

Proof  of  the  Lie  Formula 

To  prove  the  Lie  formula  (5.8)  for  finite  dimensional 
matrices,  we  use  the  identity 


i 


I 

i 


lie 


C+D 


/ C D \n 

-H' 


n-i  k C+D  CD 

„ v 1 1 C+D  1 1 — |,  n nn,, 

<_  l e"  "n  ||e  -ee|| 

k-0 


n-l-k 


./.llcll  + IlDllj  " 

C+D  C D 

< n ||e  n - enen||  exp((|  |c|  | + | |d|  | ) (1-1/n)  J 
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sace 


On  the  other  hand. 


* "(?)  


(n  -*■  <») 

for  any  K > j||CD-DC||  , proving  (5.8). 

Eq.  (5.8)  is  also  true  for  certain  infinite  dimensional 
matrices  (operators)  . This  deep  result  known  as  the  Trotter 
product  formula  is  very  useful  in  the  modern  theory  of 
partial  differential  equations. 


C+D  C D 

i,  n n n>,  ^ 

||e  - e e ||  < 


so  that 


C+D 


/ £ 5. \n 

- H 1 
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6.  Spectral  Methods  Using  Fourier  Series 

Fourier  series  are  appropriate  to  solve  problems 
with  periodic  boundary  conditions.  With  periodic  boundary 
condi tions,  a stable  spectral  method  based  on  Fourier  series 
is  usually  accurate  and  efficient.  On  the  other  hand,  when 
Fourier  series  are  used  to  solve  non-periodic  problems 
(including  problems  having  periodic  initial  conditions 
but  whose  evolution  operators  violate  periodicity) , 
stability  is  not  enough  to  ensure  convergence  to  the  true 
solution  of  the  problem.  An  example  of  the  latter  effect 
was  given  in  Example  1.3.  In  this  section,  we  investigate 
the  stability  and  convergence  of  spectral  methods  based  on 
Fourier  series. 

Example  6.1s  Constant-coefficient  hyperbolic  equation  with 
periodic  boundary  conditions 

Consider  the  one  dimensional  wave  equation 

ufc  + ux  = 0 (6.1) 

u (x,  0)  - f ( x) 
with  periodic  boundary  conditions 

u(0 , t)  * u(l,t)  . 


■ 
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Since  collocation#  Galerkin  and  tau  methods  are  identical  in 
the  absence  of  essential  boundary  conditions  (see  Sec.  2), 
let  us  analyse  the  Fourier-col location  or  pseudospoctral 
method.  We  introduce  the  collocation  points 
xn  - n/2N  (n  - 0, . . . , 2N-1)  and  the  vector  notation 
u *(Uq' * * ,,U2N-1)  where  un  " u^xn^  • The  collocation 
equations  that  approximate  (6.1)  can  be  written  as 


9u 

Tt 


-l 

C D C U , 


(6.2) 


where  C and  D are  2N  * 2N  matrices  whose  entries  are 


C.  - -i—  exp (-2ni (k-N) x. ) , 
/ 2N 


(6.3a) 


Dk,  - -!.i  *•  «k(  . (t..3b) 

where  k*  ■ k-N  (i  < k < 2N-1  ) and  k'  ■ 0 if  k * 0 . A 
simple  derivation  of  (6.2)  is  obtained  by  observing  that 
Cu  gives  the  Fourier  coefficients  of  the  collocation  projection 
Pu  of  u(x)  . Thus,  DCu  are  the  Fourier  coefficients  of 
- 5^  Pu  and,  finally,  C-1  DCu  gives  the  collocation  projection 
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of  - JL  Pu  which  is  - P Pu.  The  matrix  C is  a unitary 
jx  3 x 

matrix  so  C*  - C_1 , and  the  matrix  D is  skew-Hermitian  so 
D*  - - D.  Therefore,  C-1DC  is  skew-Hermitian  so  that 

||explC_1D  Cl  1 1|  - 1 . (6.4) 


This  proves  that  the  Four ier-col location  method  is  stable  for 
(6.1).  The  results  of  this  example  can  be  generalized  to  a 
general  system  of  constant  coefficient  hyperbolic  equations. 

Example  6.2j  Variable-coefficient  hyperbolic  equation  with 
periodic  boundary  conditions 

Consider  the  system  of  equations 

u ♦ A(x)  u - 0 0 < x < 1 

w A “ 


with  periodic  boundary  conditions  u(0,t)  - u(l,t)  and  periodic 
inhomogenity s A(x)  ■ A(x+1)  for  all  x . Here  u(x)  is 
a vector  of  m components  and  A(x)  is  an  m x m matrix. 

If  we  assume  that  A(x)  is  a symmetric  matrix  and  that 


3A  £ a I 

5x 


(6.5) 
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for  some  finite  a , then  the  Fourier-Galerkin  method  is 
stable.  To  show  this,  we  denote  by  uN  the  N-term  Fourier- 
Galerkin  approximation  of  u.  Then  using  integration  by  parts 
we  obtain 


1 1 1 
ar  / un  uNdx  = f US,A+A*)x  uNdx  i2“/  v»dx  • 


Therefore, 


J u*  (x,  t)  uN  (x,  t) dx  <_  uN(x,  0)  uN(x,0)dx 


which  proves  stability. 

Condition  (6.5)  is  not  sufficient  to  ensure  stability  for 
the  collocation  method.  Consider  the  scalar  equation  (m  = 1) 


u.  = r(x)  u 
t x 


0 < x < 1 


(6.6) 


u(0,t)  = u(l,t)  . 


If  we  impose  the  additional  restriction  that  r(x)  is  non-zero 
within  0 < x <_  1,  then  we  can  prove  that  the  collocation 
is  stable.  We  must  show  that  exp(RC*DCt)  is  stable  where  C and 
D are  given  by  (6.3)  and  R is  the  matrix  with  entries 

Rij  = r(xi)  6ij  * 


The  matrix  R can  be  identified  as  the  Liapounov  matrix  H, 

r 

invoked  in  (5.7)  and,  therefore,  the  method  is  stable; 


-100- 


R-1 ( RC*DC)  + (C*D*CR* ) R_1  - 0.' 


In  fact,  following  the  proof  of  the  main  result  in  Sec.  5, 

|| exp (RC*DCt)  1 1 2 < ||R||  Hr”1  II  1 “ax  | r (x) | /min  |r (x) | , 

0<x<l  0<x^l 

proving  stability  for  N « . 

If  r(x)  has  a zero  within  0<x<l,  collocation  with  Fourier 
series  may  lead  to  instability.  For  example,  if  N ■ 2,  the 
eigenvalues  of  RC  DC  are  0,0,  */-(r0+r2)  (r^-»-r3)  where  r^  - r(x^), 
so  there  are  growinq  modes  if  (r^+rj) (r^+r^)  < 0.  In  some  cases, 
these  modes  may  have  large  growth  rates.  One  way 
to  limit  the  growth  rate  of  these  modes  is  ts  rewrite 
(6.6)  as 

ut  + y (r(x)u)x  + y r(x)ux  - y u - 0 (6.7) 

Now  Fourier-collocation  gives  the  matrix  equation 

ut  + (»jC*DCR  + SRC*DC  - Q)u  - 0 
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where  Ojq,  * - j r,*xk)l!1ke  * The  first  two  matrices  on  the 
right  side  add  up  to  a skew-Hermitian  matrix.  Also,  if  (6.5) 
holds  for  r(x)  then  Q <_  ^ al.  Therefore,  we  obtain  the 
inequality 


d i | 2 | | 2 

at  lul  i a M 


Thus,  we  see  it  is  possible  to  bound  a priori  the  growth  of  modes  in  the 
Fourier-collocation  method  for  variable  coefficient  problems  with 
periodic  boundary  conditions. 

On  the  other  hand,  for  problems  with  non-periodic  boundary 
conditions,  Fourier-spectral  methods  can  produce  wrong  solutions 
even  when  they  are  stable.  This  is  illustrated  by  Example  1.3 
which  we  now  study  more  carefully. 


Example  6.3:  Hyperbolic  equation  with  non-periodic  boundary 
conditions 


Consider  the  problem  (1.7) 


?u(x,t)  3u(xtt)  „ 

at  5x  xt 

(0  < 

x < n 

t > 0)  , 

u(0,t)  = 0 

(t  > 

0) 

(6.8) 

u(x, 0)  » 0 

(0  < 

x v n) 
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The  solution  is 


u(x,t)  = xt  . 

If  we  attempt  to  solve  (6.B)  by  Fourier  sine  series  using  the 
Galerkin  procedure  we  obtain 

N 

Ujj  ® I an  sin  nx  (6.9) 


dan 

dt 


a N 

~ I 

71  m=l 
m+n  odd 


nm 


2 2 
n -m 


am 


in;  11  en 


(6.10) 


where  e = 0 if  n is  even  and  e = 1 if  n is  odd. 
n n 

It  is  easy  to  verify  that  the  above  approximation  is  stable. 
If  we  write  (6.10)  in  the  form 

i!  = ANa  + f 


where  a = (a1#...#aN),  £ = (f x , . . . , fN)  , fR  = | [(-l)n+1+  2ten/TrJ, 
then 

+ - 0 • 

Thus,  | lexptA^t) ||=1  for  all  N and  t. 

In  Figs.  6. 1-6.4  we  plot  the  solution  of  (6.9-10)  at 

t = 1 for  N = 25,  50,  75,  100  . It  is  apparent  that  uN(x,l) 

does  not  converge  to  the  exact  solution  xt  at  t = 1 as 

N -*■<*>  . Instead,  u„  for  N even  appears  to  be  converging  as 

N 
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-L-J 


' - 


Fig.  6.2.  Same  aa  rig.  6.1  except  N-50,  t-1 


N to  the  function 


( 

xt 

x > t 

U = 1 

(6.11) 

even  j 

n (x-t) +xt 

X < t , 

for  t < tt^  while  uN  for  N odd  appears  to  converge  to 
the  function 


xt 

IT  (t-x)  +Xt 


X > t 
X < t 


(6.12) 


for  t < tt  . The  results  plotted  in  Fig.  6.5  for  u10Q(x,t=2) 
are  also  consistent  with  convergence  to  the  wrong  solution 
(6.11).  Notice  that  the  approximations  u^X/t)  plotted  in 
Figs.  6.1-5  all  exhibit  a large  region  of  nonuniform  con- 
vergence near  x = 0 and  x = u and  that  the  errors  in 
the  interior  of  the  interval  0 < x < tt  decrease  with  N 
roughly  like  1/  . 

The  origin  of  the  divergence  of  (6.9-10)  from  the  exact 
solution  to  (6.8)  is  not  instability;  rather,  the  divergence  is 
due  to  inconsistency.  Since  ||exp(ANt)  ||  = 1,  the  method  is 

stable.  To  show  that  it  is  not  consistent  we  estimate  the 
truncation  error  in  the  L 2 norm. 


-i  np- 


i 


— — 


ii 


N 


|Lu  - Lnu| | , 


for  u = xt  where  L„  = P„LPM  and  P„  is  the  Galerkin 

N N N N 

projection  operator  and  L = - 9/3x  . This  error  can  be 
bounded  from  below  by 


£n  = IIlu-pnlu  + pnlu  -pnlpnu| 


1 ||pnl(i-pn)u||  - ||(i-pn)lu|| 


However,  ||  ( I— P^) Lu ||  ■+  0 (like  1/^N)  as  N -*  °°  because 

this  norm  is  just  the  error  in  the  Fourier  sine  series  ex- 
pans ion  of  Lu  = - ^ xt  = t . Therefore,  if  we  can  show 
that  ||PNL(I-PN)  u||  does  not  approach  zero  as  N -*•  <*> 

then  (6.9-10)  is  not  consistent. 

To  estimate  ||  PNL  ( I-PN)  u ||  we  proceed  as  follows. 
Since 
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This  completes  the  proof 


— 


for  suitable  constants  C and  C^. 
that  | |Lu  - LNu|  I does  not  approach  zero  as  N ®. 

Blair  Swartz  (private  communication,  1976)  traces  the 
inconsistency  of  (6.9-10)  to  the  incompleteness  of  the  set 
of  functions  (L(sinnx)  = - n cos  n x,  n=l,2,...}.  This  set  of 
functions  is  made  complete  by  augmenting  the  set  by  the  function  1 . 
Whereas  u may  be  well  approximated  by  a function  uN  of 
the  form  (6.9),  Lu  may  not  be  well  approximated  by  the 
function  Lu^.  In  fact,  if  | |Lu  - LuN|  |-*-0  as  N ® , 
then 

H 

/ (Lu  - Lu^j)  dx  -*■  0 (N-*-®)  . 

Since 

IT  IT 

/ Lun  = - / l na  cos n x dx  = 0, 

0 0 n 

Lu  may  be  well  approximated  by  Lu^  only  if 

IT 

0=  / Lu  dx  = u(0)  -u(tt), 

0 

which  is  generally  not  true. 

As  shown  in  Figs.  6.1-5,  u^  (x,t)  does  converge  to  xt 
as  N -*■  ®.  The  analysis  given  above  provides  no  clue  to  the 
fascinating  way  in  which  the  method  achieves  this  divergence. 

N 

There  is  no  indication  of  the  ’error'  wave  (-1)  ir(x-t)  that 
appears  in  (6.11-12)  and  propagates  with  speed  1 across 
0 < x < it  . It  seems  that  the  complete  mathematical  analysis 
of  the  divergence  of  (6.9-10)  is  difficult  and  we  do  not  now 
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have  a justifiable  argument  to  demonstrate  convergence  of 

to  u and  u given  by  (6.11-12)  as  N -*■  « through 

even  odd 

even  and  odd  values,  respectively. 

In  the  next  example  we  will  show  that  it  is  not  simply 
the  presence  of  boundary  conditions  but  rather  the  non-periodic 
nature  of  the  problem  that  causes  the  divergence  of  the 
Fourier-spectral  methods. 


Example  6 . 4 Non-periodic  boundary-free  problem 


Consider  the  problem 


3U  4-  (v  n\  9u 

Jt  + (X'I) 


u(x,0) 


(0  < x < n) 


(6.13) 


The  problem  is  well  posed  without  specifying  any  boundary  con- 
dition. However,  since  the  solution  is  given  by 


u(x,t)  = f(y  + (x-^-)e_t) 


(6.14) 


it  is  clear  that  the  solution  is  not  periodic  in  x . Since 
r(x)  * x - ^ has  a bounded  derivative,  it  follows  from  Example 
6.2  that  Fourier-Galerkin  approximation  to  (6.13)  is  stable. 
Nevertheless  it  is  not  convergent  as  shown  by  the  results 
plotted  in  Figs.  6.6-8  for  f(x)  = sin x and  N * 5,  10,  and 
20  retained  terms  in  the  Fourier  sine  series. 
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Polynomial  Subtractions  for  Non-Periodic  Problems 

There  is  a method  that  can  be  used  to  ensure  that  Fourier 
series  yirld  convergent  results  for  non-periodic  problems. 

The  idea  is  to  express  the  solution  as  the  sum  of  a low-oi'der 
polynomial  and  a Fourier  series;  the  polynomial  is  chosen  so 
that  the  Fourier  series  converges  rapidly  as  suggested  originally 
by  Lanczos  (1956,1966)  . The  method  has  been  used  by  Orszag 
(1971c)  and  Wengle  & Seinfeld  (1977)  to  solve  problems  with 
non-periodic  boundary  conditions.  We  illustrate  it  here 
for  the  problem  discussed  in  Example  6.4. 

Example  6.5  Polynomial  subtractions  applied  to  Fourier  series 

The  Fourier  sine  series  expansion  of  the  exact  solution 

u(x,t)  to  (6.13)  converges  slowly  because,  in  qeneral, 

C 

u(0,t)  ft  0 and  u(n,t)  ft  0 . This  slow  convergence  of 

the  Fourier  series  of  the  exact  solution  implies  that  Galerkin 

approximation  is  inconsistent,  as  shown  using  the  methods 
l 

of  Example  6.3.  In  order  to  avoid  slow  convergence  or  even 

divergence,  we  proceed  as  follows. 

We  seek  the  solution  to  (6,13)  as  the  sum  of  a linear 

* 

polynomial  and  a Fourier  series: 

cc 

u(x,t)  * b(t)x  + c(t)  (it— x)  + 7 a (t)sinnx  (6.15) 

n=  i n 

where  b(t)  and  c(t)  are  chosen  to  ensure  that  an(t)  -*  0 

rapidly  as  n ■*  ® . Substituting  (6.15)  into  (6.13)  gives 
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1 1 ) x + c'ltKn-x)  + v a' (t)sin  nx  = (i-x)lb(t)-  c(t)] 


+ l a ,,  ( t)  sin  nx 
n=l  n 


(6.16) 


where 


a„lt> 


V 

m=  1 n “ - m 

n+m  even 

n*m 


2 run  1 

rr?  am  + I an 


(6.17) 


are  the  Fourier  sine  coefficients  of  ( 


3 

x)  ^ E sin  nx  . 


If  we  knew  u(0,t)  and  u(’>ft)  we  could  set  b ( t)  »u(n , t)/"  and 
c(t) »u(0, t) / n ; with  this  choice,  the  Fourier  sine  series  in 
(6.15)  does  not  exhibit  the  Gibbs  phenomenon  and  (t) =0 (1/n^) 
as  n*~.  However,  the  boundary  conditions  on  u are  not  known 
as  part  of  the  specif ications  of  the  problem  (6.13).  Therefore, 
we  must  solve  for  b(t)  and  c(t)  directly  from  the  differential 


equal  ion . 


Equating  coefficients  of  sin  nx  in  (6.16)  gives 


vUl  ^ . •»  A 

jtn  = Ic'-b'+c-bl^-l)"  1 + (b-c-2c ’ ] ~ en  + aR  (n=l,...)  (6.18) 


where  o = 1 if  n is  odd,  0 if  n is  even;  here  we  use  the 
Fourier  sine  series  expansion  of  1 and  x: 


- 1 1 8- 


n+l  sin  nx 

" • 
n 

3 

Also,  if  b ( t)  and  c(t)  are  chosen  so  that  an-0(l/n  ) 

as  n-*®,  then  the  Fourier  series  £an  sin  nx  may  be  differentiated 

termwise  so 


x = 2 l (-1) 
n=l 


Using  these  results  and  setting  x *=tt  and  x * 0 in  (6.16) 
gives,  respectively, 


db 

dt 


j (c-b) 


\ I l-l>" 

n=l 


(6.19) 


I 1 


GalerKxn  approximation  repruuutt-o  uic  cvjUMw^v.w  ... 

* 0 for  n «=  N+l , N+2,  ... 

The  above  derivation  suggests,  but  does  not  prove,  that 

a (t)  -*■  0 sufficiently  rapidly  as  n -*■  °»  that  inconsistency 
n 

problems  are  avoided.  The  exact  solution  of  (6.13),  which 
satisfies  (6.18-20)  with  N = 00  , does  satisfy  an  * 0(l/n3) 
as  n-*°°  . However,  the  Galerkin  approximation  with  finite  N 
does  not  yield  such  a rapidly  converging  result.  In  fact, 
estimates  like  those  given  in  Example  6.3  show  that 

||Lv  - Lnv||  = O(-^y-j)  (N  -*  «>)  (6.21) 

N 

where  v satisfies  v(0,t)  = v(n,t)  = 0 and  L = (^-x)-^  . 

Since  the  Galerkin  approximation  (6.18)  is  stable  (see  Example 
6.6),  we  expect  that  the  errors  in  the  Galerkin  approximation 
(6.18-20)  are  of  order  N~3//2  for  fixed  t. 

The  above  prediction  has  been  tested  numerically.  In 
Table  6.1  we  list  for  various  N the  maximum  errors  in  the 
approximation  obtained  by  solving  (6.18-20).  A plot  of  the 
error  u fx,t)  - u(x,t)  vs  x for  N = 30,  40  at  t = .5  is 

N 

given  in  Fig.  6.9  *10  . 

In  the  next  example , we  prove  that  the  method  of 
polynomial  subtraction  used  in  Example  6.5  is  stable. 

Example  6.6.  Proof  of  stability  for  polynomial  subtractions 
It  is  not  obvious  that  the  approximation  (6.18-20)  is 
stable.  Fourier  series  approximation  without  polynomial  subtractions 
are  stable  but  not  consistent  (see  Example  6.4).  On  the  other  hand, 
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Table  6.1 


N 

eN«  maxlu^tx,  t».  5)  - u(x,  t *.5)| 

N3/2e 
n eN 

5 

4.19  (-3) 

4.7  (-2) 

10 

2.13  (-3) 

6.7  (-2) 

15 

1.13  (-3) 

6.6  (-2) 

20 

8.28  (-4) 

7.4  (-2) 

25 

5.76  (-4) 

7.2  (-2) 

30 

4.70  (-4) 

7.7  (-2) 

35 

3.64  (-4) 

7.5  (-2) 

40 

3.13  (-4) 

7.9  (-2) 

Table  6.1.  Errors  in  the  polynomial -subtracted  Fourier 
series  approximation  u^Xft)  given  by  (6.22)  and 
(6.18-20)  for  the  problem  (6.13)  with  f(x)  = sin  x 
for  t«.5.  Observe  that  the  errors  appear  to  decrease  as 


in  the  polynomial- 


the  approximations  obtained  by  polynomial  subtractions  are  consistent 
as  shown  by  (6.21),  but  their  stability  remains  to  be  shown. 


To  demonstrate  stability  of  (6.18-20),  we  reformulate 
these  equations  in  terms  of  u..(x,t)  defined  by 

N 

N 

u„(x,t)  = b(t)x  + c(t)(n-x)  + Y a (t)  sin  nx  . (6 

N n=l  n 

In  terms  of  u„(x,t),  (6.18)  is  equivalent  to 

N 

sin  nx  dx  = 0 (n=l,...N)y  (6 

while  (6.19-20)  become,  respectively, 

« 0,  (6 

x = n 


% 


TT  T (x_  3°  dx 


/ 

0 


3un 

dt 


+ (x-  i) 


% 
3 x 


+ (x  . S) 

3t  1 r 3x 


= 0, 


(6. 


x = 0 


Multiplying  (6.23)  by  n an»  suming  from  n = 1 to  n = N,  and 
noting  that 


3x2 


N 


n=0 


n a sm  nx  , 
n 


.22) 


.23) 


.24) 


25) 
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we  obtain 


L. 


IT 

/ 

0 


3un  + 

TT  + 


, n \ 
(x-  ») 


3un 

V Tx” 


92u 


N 


dx  **  0 


(6.26) 


9x‘ 


Integrating  (6.26)  once  by  parts  and  using  (6.24-25),  we  obtain 


n 

/ 

0 


9 

YU 


9u, 


N 


TT 


+ (x  - i) 


9x 


£ 


dx  - 0 


Therefore, 


9 

9t 


, /3UA 
0 \ ‘ / 


dx  = - 2 


? /3Un\  2 „ 7 , IT.  9 /9Un\ 

/ \-sijy  dx  • l lx‘  2*  n 


dx 


Integrating  the  second  integral  on  the  right  once  by  parts  gives 


3 

Yt  K 


2 IT 

dx  = - / 
0 


fe) 


dx  - 


(S)'  I • (5) 


X=TI 


*x*0  -J 


so  that 


9 

Yt 


dx. 


Thus,  we  obtain  the  stability  estimate 

9uN(x,t)  1 2 


dx  < e 


-t 


/ f-^-1 


dx 


(6.27) 


The  bound  (6.27)  shows  the  stability  of  (6.18-20). 


Examples  6.5-6  suggest  that  by  subtracting  polynomials  of 

higher  and  higher  degree  from  u(x,t),  the  residual  Fourier 

series  can  be  made  to  converge  faster  and  faster.  Subtracting 

a linear  polynomial  as  in  (6.15)  gives  Fourier  approximations 

-3/2 

with  errors  of  order  N as  ; subtracting  a quadratic 

polynomial  gives  Fourier  approximations  with  errors  of  order 
N and  so  on.  In  the  limit  we  disperse  entirely  with 
Fourier  series  and  obtain  a rapidly  converging  polynomial 
approximation.  The  convergence  theory  of  these  polynomial 
spectral  approximations  is  discussed  in  the  next  two  sections. 


L 


1 
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7 . Applications  of  Algebraic-Stability  Analysis 


The  main  result  of  Sec.  5 does  not  provide  us  with  a 
systematic  way  of  constructing  the  family  of  Liapounov 

matrices  necessary  to  prove  algebraic  stability.  In  general, 
these  matrices  are  difficult  to  find.  However,  there  are 
several  problems  for  which  they  can  be  found  directly  from 
the  differential  equation. 

It  is  very  easy  to  construct  Liapounov  matrices  for  Galer- 
kin  approximations  to 


9u 

rE 


a LU 


where  L is  a semi-bounded  operator  on  the  Hilbert  space 
We  say  that  L is  semi-bounded  if 


L + L*  < a I 


(7.1) 


for  some  constant  c»  > where  L*  is  the  adjoint  of  L defined 
with  respect  to  the  Hilbert  space  inner  product  ( , ) . If  L 
is  semi-bounded 


_d_ 

dt 


(u,u) 


< a (u,u)  , 


(7.2) 


so 


(u(t)  ,u(t)  ) - eat  (u  (0)  , u ( 0 )) 
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•• 1 1111  - rm 

i 

and  the  'energy'  (u(t),u(t))  grows  at  most  exponentially  with 
t . 

If  an  energy  estimate  of  the  form  (7.2)  exists,  then  Galerkin 
approximation  based  on  the  Hilbert  space  inner  product  ( • , • ) is 

: 1 

atable  (and,  hence,  algebraically  stable).  The  Liapounov 
matrix  Hn  may  be  chosen  to  be  the  N x N identity  matrix 
IN  . In  fact,  it  follows  from  the  Galerkin  equations  (2.6-7)  that, 
if  f = 0,  then 

A (VV  = (UN'  (L+L*)uN)  - 01(UN'UN) 

C-S* 

Thus, 


I 


. 


(u^t)  ^(t))^  eat(uN(0)  ,1^(0)) 

Since  uN(t)  = exp (LNt)uN (0)  for  all  uN(0),  it  follows  that 

llexP(LNt)||  < exp(iat)  so  stability  is  proved.  T^e  reader  is 
reminded  that  with  stability  established,  the  theory  of  Sections 
4 and  5 proves  convergence  for  consistent  schemes. 

Example  7.1:  Semi-bounded  Galerkin  approximations 

The  above  construction  establishes  stability  and  thus  con- 
vergence for  a wide  variety  of  Galerkin  approximations.  Among 
these  stable  Galerkin  approximations  are: 

(i)  Solution  of  any  problem  ufc  = Lu  that  is  semi-bounded 
in  1*2  (-1,1)  by  means  of  Legendre  series.  For  example, 
ufc  + ux  = f(x,t)  with  u(-l,t)  =0  is  stable  (and  convergent) 


j 
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when  solved  by  Legendre-Galerkin  approximation.  For  our  argument 
to  be  complete  it  is  necessary  to  verify  that  the  Legendre- 

Galerkin  approximation  to  this  problem  is  consistent.  This  is 
done  as  follows . 

We  write 

||lu-pnlpnu||  < ||  (i-pn)Lu||  + ||pnl(i-pn)U||  . 

The  first  term  on  the  right  goes  to  zero  as  at  a rate 

governed  solely  by  the  smoothness  of  Lu  ; it  measures  the 
error  in  the  N term  Legendre-Galerkin  expansion  of  Lu  . 

The  second  term  is  estimated  as  follows.  Set 

oo 

L(I-Pn)u  = l an*n<x) 
n=i 


where  { <f>n>  are  normalized  Legendre  polynomials.  If  L is 
a finite-order  differential  operator  so  L*  is  also  a finite- 
order  differential  operator  (for  example,  L*=3/3x  if  L = -3/3x), 
then 

= (V  L<I-VU) 

= (L**n,  u-Vu)  • 

Thus, 

I an  I i l|L%nll  ||(I-VUII 

= 0 (nA/NB)  (n-+«  ; N->»  ), 

where  A depends  only  on  L (A  = 3/2  if  L - -3/9x  and  <J>n  is  a 
normalized  Legendre  polynomial)  and  B depends  only  on  the 
•smoothness  of  u (B  is  arbitrary  if  u is  infinitely  differentiable)  . 
Thus, 


||PnLU-pn)u||  * 0 

faster  than  any  power  of  1/N  if  u and  all  its  derivatives 
are  smooth.  This  proves  consistency.  This  kind  of  proof 
extends  to  a wide  variety  of  the  examples  to  be  discussed  in 
Sects.  7 and  8,  but  will  not  be  repeated. 

(ii)  Solution  of  u * xu  with  the  boundary  conditions 

U A 

u(±l,t)  * 0 is  a well  posed  problem  in  the  Chebyshev  inner 
product 

1 

(u.v)  • I al»)YW-  a*  . 

-1  (1-xV 

In  fact,  if  L * x 3/3x  , and  u is  differentiable  and 
satisfies  u(il)  - 0 then,  by  integration  by  parts, 

x 1 2 --  2 < 

(u,Lu)  - f1  x(l-x2)'*  u u dx  *=  - / (1-x  ) 2u  dx  - 0 . 

-1  -1 

Thus,  Galerkin  approximation  to  the  problem  is  stable  using 
Chebyshev  polynomials. 

(iii)  Solution  of  ut  + ux  “ ® (0  < x < »)  with 

u(0,t)  * 0 is  a well  posed  problem  in  the  Laguerre  inner 
product 


(u,v)  = / u(x)v(x)e  xdx  . 

0 

» 

In  fact,  if  u(0,t)  * 0 then,  by  integrating  by  parts. 


- / uu  e Xdx 
x 

0 


Similarly,  the  problem  ufc  = uxx  (0  x < »)  with  u(0,t)  = 0 
is  also  stable  in  the  Laguerre  norm. 

(iv)  Solution  of  = -xux  (-®  < x < «,)  is  well 

posed  in  the  Hermite  inner  product 

°°  _x2 
(u,v)  = / u (x)  v (x)  e dx  . 


In  fact. 


w 4 

u,u)  = -2  / x e x 


3t 


uu  dx 

X 


so  that  integration  by  parts  gives 


a 

at 


(u,u) 


/ u2e”x  ( 1- 2x2 ) dx  < (u,u) 


X 2 

where  we  assume  that  u <<  /x  exp  (-^  x“)  as  I x | -*  a> 

(v)  The  heat  equation  ufc  = uxx  with  u(±l,t)  =0  is  semi- 
bounded  in  the  Chebyshev  norm.  In  fact,  if  u is  differentiable 
for  | x | <_  1 then 


t - 


2v”7  j 

x ) uuxx  dx 


(1-x2)  ? UU  I 


[U(l-x2)  2lxux  dx 


The  first  term  vanishes  because  u is  a polynomial  in  x and 
therefore  u(±l)  = 0 implies 


2 l/2 

(!-x  ) x = ±1 


= 0 


The  integral  term  on  the  right  is 


(u(l-x  ) )x  ux  dx 


1 

-1 


2 2 2 2 2 ^ 

(u(l-x  ) ) (u(l-x  ) ) (1-x  ) dx 

X x 


r £ & r<»u-x2) 


1 7 

2 l2 


x(l-x  ) dx 


2 | 2 
(u(l-x2)  )x  (1-X2)  dx 


+ j u2x(l-x2) 


3 

2 ,1 


-if  . 

* J-i 


2 2, 

(1-x  ) 


< 0 


and  therefore 


1 2 


_d_  [ u 

dt  l-i  jr 


dx  < 0 
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(7.3) 


In  the  next  two  examples  we  generalize  the  proofs  of 
stability  and  convergence  for  Galerkin  approximations  given  in  Example 
7.1  to  show  the  stability  and  convergence  of  tau  approximations. 


Example  7.2:  Semi-bounded  tau  approximations 


(i)  Cohsider  the  equation 


3u  „ 9u 
— = x — 


3t  3x 


with 


u (±1 , t)  = 0 


It  was  shown  in  Example  7.1 (ii)  that  if  L=  x3/3x,  then 


L + L*  < 0 


in  the  Chebyshev  inner  product.  If  we  seek  the  solution  as  the 


truncated  Chebyshev  series 


l a T 
*•  n n 


by  the  tau  method,  then  Uj^  satisfies  exactly  the  equation 


3u.. 

- X — -Jl  * x (x)Tn(x)  + tm_i  (t)TN-l  <7-4) 

Dt  3x  N 


N N— 1 

Equating  coefficients  of  x and  x on  both  sides  of 


(7.4  ) , we  obtain 


■■ 
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aN  “ NaN  TN 


aN-l  “ <N_1)aN-l  TN-1 


since  Tn  = 2n~1xn  - n2n_3xn_2  + 


. Therefore, 


I 

I 


1 


I 


(“n  Tf)  = ( (L+L*)UN'UN)  + IaN~NaN]aN 
+ faN-l“  (N_i)aN-lJ  aN-l 


so  that 


(7.5) 


i ft  ='(L+1?)un,un)  - Na^  - (K-luJ-j  i o. 


Since 


N 


(UN'UN}  ^ aN  ' 
n=0 

the  above  inequality  is  equivalent  to 


■ ; 

3 

It 

N-2 

I 

n=0 

2 < 
aN 

0 

(7.6) 

This  proves  stability: 

aN 

and 

aN-l 

are  bounded 

because  they 

are  determined  in  terms 

of 

a0' 

al'  * * 

• ,aN_2  by  the 

boundary 

conditions  u(±l,t)  = 0 

• 

For  this  example,  we  can  prove  stability  directly  from  the 
matrix  representation  of  1^.  In  fact. 
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( 7 . 7f ) 


2,  even 


In  the  tau  approximation,  the  boundary  conditions  u(±l,t) 
that  the  last  two  rows  of  the  matrix  be  replaced  by 


= 0 require 


N N-l , k 


= ("1)  / 


(7.7b) 


(L  ) = 1. 

N N,k 


(7.7c) 


If  the  boundary  conditions  (7.7b,c)  are  not  applied  then 

the  spectral  approximation  is  unstable:  without  the  boundary  conditions 

_,N. 

L has  the  eigenvalue  N (with  the  eigenvector  apj_2k  k ' 

N 


aN-2k-l  ' 01  S°  that 


To  prove  convergence  when  the  boundary  conditions  (7. 7b, c)  aie 
applied,  let  us  first  consider  an  odd  solution  in  which  n^  - 0 
if  n is  even.  If  we  assume  that  N = 2M+1  and  set 


dk  ~ a2k+l 


(0  * k < M) 


then  the  system  reduces  to 


ft ' D a 


where 


D.k*-(2k+l)5.k+  2 l (2k+l)«  j+jfk-2N  (0<  j<  M , 0<k^M) 


f wc  introduce  the  M x M transformation  matrix  S defined  by 


S..  “ 6 

jk  3k 


“ 5-,,  . ( 0^ j , 0<k<M  ) 

ik  l + 1'k  ~ • 
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then  S(D+D*)S*  is  a diagonal  matrix  with  entries 

(-4,  -4,  . ..,  -4,  -4N  - 12).  Thus,  we  obtain  D + D*  < 0, 

so  that  3 (3,3) /3t<0  which  proves  stability. 

Example  7.3.  Stability  of  tau  methods  applied  to  degree-reducing 
semi-bounded  equations 

An  argument  similar  to  that  given  in  Example  7.2  demonstrates 

stability  of  tau  methods  in  terms  of  arbitrary  orthonormal  polynomial 

bases  for  equations  — = Lu  whore  L is  semi-bounded  and  degree 

reducing:  L is  said  to  be  degree  reducing  if  for  any  polynomial 

PN  of  degree  Ny  LP^  is  a polynomial  of  degree  at  most  N - k 

where  k is  the  number  of  boundary  conditions  that  are  applied. 

N-k+2  N 

If  L is  degree  reducing,  equating  coefficients  of  x ,...,x 

in 


3u 


N 


N 


at 


= L 


u . + J" 

N L 


T <p 

n^  n 


n=N-  k+1 

implies  that  Tn(t)  = a^(t)  for  n = N-k+l,...,N;  here 


N 

uN(x,t)  = l an(t)$n(x) 

• n=0 

and  the  orthonormal  expansion  polynomial  <|>n(x)  is  assumed  of  degree  n. 
Therefore, 

' \ N 

* <Tt*UN'UN*  ” ^ anan  = (fL+L*3  uN ,uN)  - 0 

n=N-  k 


so  that 


are  determined  by 


which  proves  stability  since  — 'aN 

the  boundary  conditions  in  terms  of  «0'al"  ' ’ 'aN-k  * 

Example  7.3:  More  stable  tau  approximations 
(i)  Suppose  that 

ut  + ux  * 0 (-1  i x < 1,  t > 0) 

u (-1 , t)  = 0 

is  solved  by  tau  approximation  using  Legendre  polynomials. 
The  Nth  degree  Legendre  polynomial  uN  satisfies 


so  that 


a*  I /1  - Tn4x  < 0 

which  proves  stability. 

(ii)  Suppose  that 

ut  * uxx 
u(±l,t)  * 0 

is  solved  by  the  tau  method  using  Chebyshev  polynomials.  Since 
<\2  * < 

L = is  degree  decreasing  and  L + L - 0 (see  Example 

3x 

7.1 (v)),  the  method  is  stable. 


(iii) 


The  solution  of 


u + u = 0 (0  1 x 1 “ • t > 0) 

u(0,t)  = sin t (t  > 0)  (7.8) 

u (x,  0)  =0  (0  < x < °°) 


by  Laguerre  polynomials  is  stable  using  the  tau  method  since, 
by  Example  7.1  (iii) , L is  semi-bounded.  The  equations  of 
the  Laguerre-tau  approximation  to  (7.8)  are  a simple  modification 
of  (2.23-24).  In  Fig.  7.1  we  compare  this  tau  approximation 
with  the  exact  solution  of  (7.8)  at  t = 30  for  a 20- term 
Laguerre  expansion.  The  reader  should  compare  this  approximate 
result  obtained  by  the  tau  method  with  the  best  Laguerre  approxi- 
mation to  sin  x plotted  in  Fig.  3.12. 

In  the  next  example  we  discuss  some  ways  to  find  non-trivial 
Liapounov  matrices  {H^}  whc.*n  L is  not  semi -bounded . 

Example  7.4:  Polynomial  approximations  to  a variahle  coefficient 

hyperbolic  equation 

Consider  the  initial-value  problem 

ufc  = -xux  | x | <1 

u (x,  0)  = g (x)  (7.9) 

which  is  well  posed  without  requiring  any  boundary  conditions. 

The  exact  solution~to  this  problem  is 

u ( x , t ) = g (xe  fc) 
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i 





so  that  u(x,t)  approaches  a constant  as  t-*-°°: 


u(x,  t)  ~g(0)  (t-*-°°,  |x|<l). 


The  problem  is  well-posed  in  the  sense  that  | |exp(Lt) | | 
is  finite  for  finite  t,  where  L=  - x3/3x  and  11*11  is 

the  usual  L2  norm.  However,  I I exp  (Lt)||=  exp  (j  t)  for 

any  t because  the  function  that  extremires  ||u(t)||  subject 
to 


u(0)||=  1 satisfies  u(x,0)=  gt(x)  where 


9t(x)  = 


t/2 


lx  <e 


-t 


x >e 


-t 


Therefore,  | |exp(Lt)  | | grows  exponentially  as  t-*-°°. 

The  operator  L is  semibounded  in  the  usual  L2  norm: 

/ u2  dx  = - / x dx  = - u2(l)  - u2(-l)  + j^u2  dx 

1 

< / U2  dx, 

” -1 


so  L + L*  ^ I.  Therefore,  Galerkin  polynomial  solution  of 
(7.9)  is  stable  and  convergent.  The  Legendre  polynomial  approx- 
imation uN(x,t)  satisfies 


9u. 


N 

at 


au, 


+ X 


N 


3x 


= 0 


(7.10) 
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exactly  because  no  boundary  conditions  are  applied  and  L is 
degree  preserving.  Therefore,  Galerkin,  tau,  and  collocation 
approximations  to  (7.9)  are  identical  and  all  three  methods 
are  stable. 

In  fact,  all  polynomial-spectral  methods  applied  to 

(7.9)  satisfy  (7.10);  all  polynomial  methods  for  this  problem 

give  identical  results  and,  therefore,  they  are  all  stable  in 

the  usual  L 2 norm.  In  terms  of  the  natural  norms  for  a general 

polynomial  basis  {ip  },  i.e.  that  norm  in  which  (i|>.,<|>.)  =6.  ., 

n 1 J 1J 

the  spectral  approximation  (7.10)  is  algebraically  stable  if 
the  N x N matrix  whose  elements  are 

(HN} jk  = / (x)  ^k(x)  dx 

has  a condition  number  which  is  bounded  algebraically,  i.e., 

II  HN|  I I |Hn_1|  | = 0 (N3)  (N-«)  . 

As  an  example  of  the  complicated  behavior  of  spectral 
approximations  for  this  problem  in  norms  different  from  the  usual 
L2  norm,  let  us  consider  the  Chebyshev-L2  norm.  It  may  easily 
be  shown  that  L + L*  is  not  semibounded  in  the  Chebyshev  inner 
product.  For  example,  consider  the  trial  function 


-141- 


* 


then 


((L+L  )v,v)  = -(xv  ,v)  - (v,xvx) 


T +T 
2N+1  2N- 


2N[T2N_i+...+Ti1 ,Ti  - 


N (v,  v)  . 


Nevertheless,  Chebyshev  approximation  to  this  problem  is 
algebraically  stable.  This  fact  may  be  explicitly  demonstrated 
by  construction  of  a Liapounov  matrix. 

A Liapounov  matrix  for  the  Chebyshev  approximation  to  (7.9) 
may  be  found  by  direct  examination  of  the  evolution  equation  for 
the  vector  aN  = ' (aQ, . . . ,a  ) : 


da 
r 

at 


n 


N 


~ n an  " 2 l Paf 

p=n+2 
p+n  even 


( n = 0, . . . ,N)  . (7.11 ) 


Since  a^  decouples  from  a^,...,a^  iri  (7. 11),  we  can  restrict 
attention  to  Suppose  we  define  {hn>  by 


(Hm)  = i 6 
N ]k  j jk 


d 1 j 'k  £ N). 


-142* 


Then 


the  matrix  displayed  above  has  rank  2 and  the  nonzero  eigenvalues 
are  -IN/2],  - ( (N+l)/2] . Therefore,  by  the  theory  of  Sec.  5, 

l|eL|'|t|ll>/l|HNl|||HN-1||  - /IT  . 


where  ||  • ||  is  now  the  Chebyshev  norm.  Thus,  LN  is 

* 

algebraically  stable  in  the  Chebyshev  norm  even  though 
is  unbounded  in  this  norm. 

The  qualitative  behavior  of  | | exp ( L^t ) | j as  a function 
of  N and  t is  as  follows.  For  fixed  t and 
||exp(LNt)||  = 0 (N1/4);  this  result  is  justified  heuristically 
by  following  the  argument  given  in  Sec.  5 that  led  to  (5.4) . 

On  the  other  hand  if  t > InN,  | |exp(LNt)  | | = 0(N^2)  as 
A heuristic  justification  of  this  result  is  as  follows.  Let 
u(x,  0)  » 1 for  |x|<_e,  0 for  |x|>e.  Then  the  exact  solution 

of  (7.9)  for  t>fn  1/e  is  u(x,t)  * 1 for  |x|<l,  so  ||u(x,t)||  <v 
as  €-*•()♦■  for  t>fn  1/e.  As  in  Sec.  5,  we  conclude  that 
I |exp(L„t)  I I*  0 (Ni//2)  for  t>  (nN  as  N-*«>.  (Even  in  the  usual 
L.,  norm,  | | exp(LMt)  | | = 0 (N1/2)  when  t > inN,  which  mimics  the 

i«  I*  % 

unbounded  growth  of  | | exp(Lt)|  | as  t -•■«>. ) 


8.  Constant  Coefficient  Hyperbolic  Equations 


In  this  Section,  we  discuss  the  stability  of  spectral  methods 


for  the  problem 


= o 

3t  3x 


( | x | < 1,  t > 0)  (8.1) 


with  the  initial  condition 


u (x, 0)  = f (x)  ( |x|  < 1) 


(8.2) 


and  the  boundary  condition 


u(-l,t)  = 0 


(t  > 0) 


(8.3) 


The  results  for  this  problem  can  -be  extended  to  a general 
hyperbolic  system  of  the  form 


u.  = Au 
t x 


with  characteristic  boundary  conditions , because  for  any  hyperbolic 
system  A can  be  diagonalized  by  a real  similarity  tranformation . 

The  operator  L = — is  semi— bounded  in  the  usual 
Lj (“1 » 1)  norm  when  operating  on  the  subspace  of  functions  v that 
satisfy  the  boundary  condition  v(-l,t)  =0.  In  fact 


( v , [L+L* ) v)  - -2/  v dx  = --  v2  ( 1 ) i o 
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and  therefore  Galerkin  and  tau  methods  are  stable  using 
Legendre  polynomials. 

However,  L is  not  semi-bounded  in  the  Chebyshev  norm.  To 
show  this,  we  set 

v (x)  = T2n(x)  - Tx(x)  - 2Tq (x) 
so  that  v(-l)  = 0 . In  this  case,  using  the  result 

Tk  - 2N'T2N-l+T2N-3*---+Tl1  ' 


we  obtain 

1 2 -1  3v 

( v , [L+L* ] v)  = -2  / ( 1— x ) ! ^ V dx 

1 ? -1 

= -2  J (1-x^)  7t2N(T2N_1+T2N_3+...+T1)  -Tq]  dx 

= ilti  (VfV).  (8.4) 

The  fact  L + L*  is  not  semi-bounded  is  consistent  with  the  fact 
that  exp(Lt)  is  not  a bounded  operator  for  t<2  in  the  Chebyshev 
norm  (see  Sec.  5) . However,  these  results  do  not  prove  that 
Chebyshev-spectral  approximation  to  (8.1-3)  is  not  convergent. 

In  fact,  we  shall  show  that,  while  Chebyshev-spectral  approximation 
to  (8.1-3)  is  not  stable  in  the  Chebyshev  L2  norm,  it  is  algebraically 
stable  in  this  norm. 
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in  order  to  investigate  algebraic  stability,  we  must  study  more 
carefully  the  behavior  of  the  Chebyshev  coefficients  of  the  ' 

approximate  solution 


N 


UN  * J0  an(t>  V*> 


The  differential  equations  for  th<=>  a ' c 

ans  are  given  by  (2.11) 

for  Galerkm  approximation,  (2.19)  for  the  tau  method,  and 

(2.32)  for  the  collocation  method.  As  remarked  in  Sec.  2,  all 
these  equations  may  be  written  in  the  vector  form 


is  an  (N+l)  x (N+l)  matrix. 


8 3 __ 

at  = ln  a 


where  a = (a  a , v , 

a0,ai,  • • *aN)  and  L.n  « an 

Numerical  Evidence  for  Algebraic  Stability 

Let  us  first  examine  the  behavior  of  ^ + V_  ln  Table  8-1 

we  list  the  largest  eigenvalue  of  L[J  + L„*  for  N=  10,20 100 

for  the  three  Chebyshev  methods.  This  table  indicates  that  the 
largest  positive  eigenvalue  of  + V grows  like  c„2  fQr  some 

constant  c^.  If  L N were  a normal  matrix  this  would  imply 

“at  lie  » ||  behaves  like  exp(i  CN2t).  However,  the  matrices 

ln  are  not  normal  and  therefore  the  large  eigenvalues  of  L + L » 
do  not  imply  instability.  N N 
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Table  8.1 


- • - "1 

N 

Collocation 

Tau 

Galerkin 

10 

68.84125 

, 21.4089 

72.8947 

20 

287.6920 

84.8970 

i 

296.3027 

30 

656. ; '18 

190.4908 

669.6434 

40 

1175.2124 

338.1769 

1192.9231 

50 

1843.8839 

527.9525 

1866.1433 

60 

2662.4966 

759.8167 

2689.3042 

70 

3631.0503 

1033.7690 

3662.4061 

80 

4749.5453 

1349.8093 

4785.4489 

90 

6017.9812 

1707.9375 

6058.4329 

100 



7436.3584 

2108.1534 

. ... 

7481.3579 

* 

Table  8.1.  The  larqest  positive  eigenvalue  *max  of  1*N  + 

for  the  Chebvshev-spectral  solution  of  the  one-dimensional  wave 

equation  (8.1-3).  The  Galerkin  approximation  to  this  problem  is 

given  by  the  solution  to  (2.11),  the  tau  approximation  is  given 

by  (2.19),  and  the  collocation  approximation  is  given  by  (2.32). 

2 

Observe  that  \ cN  as  N -»  <*>  where  c <-  0.75  for  the 

IHA  X 

Galerkin  and  collocation  methods  and  c * 0.21  for  the  tau 
method . 
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In  Table  8.2,  we  give  the  norms  of  the  matrices 

A 

exp Iln1  * cxp[LN  1 for  the  three  projection  methods  (Galerkin, 

collocation,  and  tau)  . The  results  indicate  that  ||exp(I<N)||  grows 

only  like  as  N (as  argued  heuristically  in  Sec. 5)  . In  other 

words  , Ln  is  algebraically  stable  (at  least  for  t~l)  . This  result 

shows  the  extreme  pess imism  of  the  energy  estimate  | |exp(LN)|  | * 

1 2 

0(exp(^-  CN  ) ) ; crude  energy  methods  may  be  very  misleading  for  non- 
normal  evolution  operators. 

In  order  to  understand  better  how  the  Chebyshev  spectral 

2 

methods  avoid  an  energy  'catastrophe'  [energy  growth  like  exp(N  t) 1 
we  have  solved  the  tau  equations  (2.19)  numerically  with  a very 
'bad'  initial  condition: 

uN(x,0)  = (Tn(x)  + 2Tn_1(x)  + (-1)N  Tq(x)  ]/ X'T  . (8.5) 


For  the  tau  method,  this  initial  condition  satisfies 


it 


('V  lV 


“(V(ln+ln 


>UN) 


t-0 


0(N2) 


(N  -+■*>) 


In  Figs.  8.1-2  we  plot  the  energy  (uN,uN)  vs  t for  N • 25 
and  N ■ 50.  It  is  apparent  that  the  initial  slope  of  the  energy 

2 

growth  is  of  order  N but  that  the  energy  does  not  maintain  this 

rapid  rate  of  growth.  Observe  that  the  region  of  rapid  growth 

is  closer  to  t * 0 for  N * 50  than  for  N ■ 25.  The  behavior 

observed  in  Figs.  8.1-2  is  not  inconsistent  with  the  fact  that 

* . Because  L.,  is 
N N 


u%,  (t  *0)  is  a 'bad'  eigenmode  of  LV1  + L 
N N 


Table  8.2 


Collocation 


Galerkin 


2.0707 

2.0003 

2,5788 

2.7932 

2.8119 

3.1903 

3.4620 

3.4857 

1 

3.8328 

4.0324 

4,0514 

I 

4.4078 

4.5222 

4.5339 

4.8630 

4.9117 

4.9855 

5.2057 

5.2961 

5.4002 

5.5262 

5.6586 

5.7770 

5.8689 

6.0282 

6.1401 

6.2526 

6.3818  ! 

« 

6.4831  1 

6.6257 

Table  8.2.  The  largest  eigenvalue  X of  exp (L^) exp (LN) 


Observe  that  *max  behaves  as  cN 


as  N > 00  where 


c * 0.6  for  all  three  spectral  methods.  The 

* 

largest  eigenvalue  of  exp  (L  ) exp  (L  ) grows  only  like 
1/2  N N 

N ' despite  the  existence  of  eigenvalues  of  LN  + L* 

growing  like  N2  (see  Table  8.1). 


non-normal  the  'bad'  initial  condition  is  not  an  eigenmode  of 

L so  that  after  evolution  from  0 to  t exp(LK,  t)  , u 'rotates' 

out  of  the  region  of  bad  modes  of  + LN*. 

The  direct  computation  of  exp[LNt]  for  t=l  is  not  enough  to 

verify  algebraic  stability  because  the  theory  of  Sec.  5 shows 

that  we  must  study  the  behavior  of  exp[LNt]  for  a complete 

time  interval  0 1 t £ T . This  may  be  done  using  the  method 

suggested  in  Sec.  5 for  the  numerical  verification  of  algebraic 

stability.  First,  in  Table  8.3  we  list  the  numerically  computed 

eigenvalues  of  1<N  . Observe  that  all  the  eigenvalues  of  LN  have 

negative  real  part. (This  result  will  be  shown  rigorously  later.) 
Therefore,  ||exp(LNt)||  ■*  0 as  t-*°°  for  fixed  N.  Thus 

the  Chebyshev  approximations  are  asymptotically  stable  in  the 

sense  that  they  remain  bounded  as  t-*00  with  N fixed. 

In  Figs.  8.3-5,  we  plot  the  L^-matrix  norm  of  exp(LNt) 

vs  t for  N=5,15,25.  Observe  that  as  t -*■<»  for  fixed  N, 

1/2 

| | exp  (L^t)  | | approaches  zero  while  it  grows  slowly  (like  N ) 

as  N^°°  for  fixed  t<2  (Note  that  growth  of  | |exp(LNt)  | | ^ 

1/2 

like  N ' as  N-*-00  is  not  inconsistent  with  growth  of 

| | exp (Ljjt)  | | 2 like  . ) Also  observe  that  the  norms  seem 

to  have  a boundary  layer  at  t=2  such  that  | |exp(LNt)  ||  ^ ■*•*> 

as  N-*-00  for  t<2  and  -+0  as  N-**»  for  t>2.  This  behavior 

is  consistent  with  the  unboundedness  of  exp(Lt)  for  t«~2  (see  (5.4)1. 

Asymptotic  stability  does  not  prove  stability  because  is 

not  normal.  The  next  step  in  the  computational  proof  of  stability 
is  to  compute  numerically  the  Liapounov  matrices  satisfying 
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Table  8.3 


\ 


t 


C 


Cl 


« 


N 

Collocation 

Tau 

Galerkin 

10 

-2.4532 

-2.999 

-1.9306 

! 

20 

-2.5932 

-3.9320 

-2.15 

30 

-2.7267 

-4.5380 

-2.32 

40 

-2.849 

-4.9918 

-2.4659 

50 

-2.966 

-5.3837 

-2.5965 

60 

-3.0824 

-5.7266 

-2.7226 

70 

-3.1985 

-6.0489 

-2.8478 

80 

-3. 3162 

-6. 3650 

-2.9738 

90 

-3.4365 

-6.6861 

-3.1017 

100 

-3.5597 

-7.0229 

-3.4335 

Table  8.3.  The  real  part  of  the  eigenvalue  of  I»N  with 
least  negative  real  part  for  the  collocation,  tau,  and 
Galerkin  spectral  approximations  to  (8.1.3).  Since  all  the 
eigenvalues  of  have  negative  real  parts,  these  spectral 

methods  are  asymptotically  stable  as  t -*•«>. 


i 
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I 


I 


(8.6) 


Vn  + 


an\ 


A good  method  to  compute  is  described  by  Bartels  & Stewart 

(1974).  In  Table  8.4  we  list  the  condition  number  of  for 

the  Galerkin,  collocation  and  tau  methods.  This  table  suggests 

3 

that  the  condition  number  of  grows  at  most  like  N as  N 

t 2 

for  the  Galerkin  and  collocation  methods  and  like  N for  the 

L 

tau  method.  Recalling  (5.11),  we  obtain 

[ 1 

p ||lexp[LNt];|  = 0 (N2  e 2)  . (8.7) 


C 


< 


< 


c 


< 


< 


L 


for  all  three  methods.  It  should  be  noted  that  (8.7)  gives  only  an 
upper  bound  for  | |exp[L^t] | | . According  to  the  theory  given  in 

Sec.  5,  this  upper  bound  can  be  sharpened  by  at  most  | | L | | = 0(N  ) 
(N-x*')  , explaining  the  origin  of  the  difference  between  the  estimate 
(8.7)  and  the  observed  behavior  N1//4  of  the  computed  L2~matrix 
norms . 

In  the  above  discussion,  we  have  given  numerical  evidence 
for  algebraic  stability  of  the  Chebyshev-spectral  methods  for 
(8.1).  We  shall  now  prove  rigorously  that  Chebyshev-spectral 
methods  for  (8.1)  are  algebraically  stable. 

Proof  of  A1 gebraic  Stability  for  C hebyshev- G a I or k in  App ro  xi mat ion 

In  the  Chcbyshcv-Galcrkin  approximation  to  (8.1),  we  represent 
the  spectral  approximation  u^  by  the  series 

N 

UN  - l an(t)  lTn“("l)n'r01  (8.8) 

n=J 


t 


5/2 

The  condition  number  of  1IK.  can  grow  no  faster  than  N ^ as 

N -12 
To  see  this,  we  note  that  (5.14)  gives  I | II  ||=  0(N“) 

w 1/4 

while  (5.13)  and  the  results  that  ||  expO^t)  | | = 0(N  ) for 

and  | |cxp(LNt)  | | ->  0 as  N *»°°  for  t*2  give  | | HN I | = 0(N1/2) 
as  N •+®. 


t 


Table  8.4 


N 

Col location 

Tau  ! Galerkin 

10 

4.1463  x 102 

i 

3.1090  x 102  4.6388  x 102 

\ 

20 

3 

3.0332  x 10 

3 3 

1.2421  x 10  3.2672  x 10 

30 

9.8746  x 103 
• 

2.7938  x 103  | 1.0464  x 104 

*' 

40 

4 

2.2940  x 10 

4.9662  x 103  2.9083  x 104 

50 

4.4220  x 101 

7.7593  x 103  ! 4.6138  x 104 

i ! 

Table  8.4.  The  condition  number  |j  | | | | | | in  the 

L 2 matrix  norm  of  the  Liapounov  matrices  for  the 

collocation  , tau,  and  Galerkin  spectral  methods  for  (8.1-3). 

For  the  collocation  and  Galerkin  methods,  the  condition 

3 

number  seems  to  grow  like  N as  N-*-®  , while  for  the  tau 

. 2 
method  it  seems  to  grow  like  N as  N->® 


Recalling  (2.34) , 


u„  satisfies 
N 


9un  + 9un 

TT  + “3x“  “ 


N T (X) 

rN(fc)  l 

N n=0  n 


(-1) 


n 


(8.9) 


N 

We  can  determine  xN(t)  by  equating  the  coefficients  of  x 
in  (8.8): 


TN(t)  = 


laN(t) 

dt 


(-1) 


N 


Let  us  now  multiply  both  sides  of  (8.9)  by  2(l-x)uN  and  integrate 

2 -1/2 

with  respect  to  the  Chebyshev  weight  function  ) • Thus, 

the  left  hand  side  of  (8.9)  becomes 


j 1 j _i  o 1 1 .1 

= ^ / (1-x)  (1-x2)  4 u^dx+  / (l-x)*(l+x)  9 dx 


^ / (1-x) (1-x2)"*  U^dx  + <l-x)*(2+x)"*  ! 


+ / u*[4(l-x)“*(l+x)“*  + i (1-x) ^ (1+x) ”^]  dx 


(8.10) 
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The  boundary  term  in  the  last  expression  vanishes  because  uN 
is  a polynomial  satisfying  uN(-l)  = 0 . Also, 


<1_xluN  * (1-*>  lVT„-(-1>nV-?,VV,-1>nT0> 


n=l 


n=l 


- ",  “n<  7 <T„+1  + Vl>  - (-1>n'rl1 

n=l 


(0.11) 

The  first  and  third  sums  on  the  right  in  (8.11)  are  orthogonal 
to  the  right  side  of  (8.9) . The  inner  product  of  (l-x)uN 
with  the  second  sum  on  the  right  in  (8.9)  gives 


- £ a 


N 


da..  j 

N , d 

dt  " 1 dt 


N 


(8.12) 


Combining  (8.10)  and  (8.12),  we  obtain 


1 2 
/ (1-x)  (1-x  ) 

-1 


-i  2 

Si 


dx  + 


j A 
1 dt 


aN 


^ 0 


(8.13) 


This  inequality  proves  that  uN  is  stable  in  the  new  norm  defined 
in  (8.13): 
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(8.34) 


[1 


it 


1 


1 1 u i r = / (i-x)  (i-x^) 

-i 


c2)-1/2  | u ( x)  | 2 dx 


It  remains  to  prove  that  the  norm  defined  by  (8.14)  i: 


algebraically  equivalent  to  the  usual  Chebyshev-L2  norm.  That 


is,  we  must  show  the  existence  of  two  functions  c^  (N)  and  c2(N) 


such  that  for  every  Nth  degree  polynomial  u 


N 


’i  / 


dx  <_  J — 


1 (l-x)u 


N 


1 u 


-1  /l-x*  -1  A-x' 


dx 


< < c / — d x (8.15) 

“ 2 - 1 /T=xT 


where  l/o  (N)  and  c2(N)  grow  at  most  algebraically  as 


The  second  inequality  in  (8.15)  holds  with  c2(N)  = 2 because 


l-x<_?.. 

The  first  inequality  in  (8.15)  is  more  difficult  to 
establish.  By  the  mean-value  theorem, 


L /— » V * «-v  [ dx 

- / i-x  -i  4.^ 


1 2 
x u 


N 


(-KV1) 


However  this  does  not  prove  the  required  inequality  because  it  is 
not.  clear  that  1/ (1-f^)  is  bounded  algebraically  as  for  all 

polynomial s. 

To  establish  the  first  inequality  in  (8.15)  we  use  a different 
approach.  We  substitute  the  Chebyshev  polynomial  expansion 


u 


N 


N 

} a T 
n=0  n n 
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and  obtain 


2 

it 


(l-x)u 


N 


-1 


dx 


/l-x: 


2aoai 


N 


♦ l 

n=l 


2 

a 

n 


N 


1 L 

* n=2 


(an  an-l  + an  an+l> 


(a0’"aN) 


H 


N 


T 


whore  II  is  the  symmetric,  positive  definite,  (N+l)  x (N+l) 

N 

tridiagonal  matrix  whose  elements  are 

if  j = k 
if  j = k-1 

(8.16) 

if  j = k+1 
otherwise , 

wnore  co  = cn  = i it  n > u.  To  complete  the  demonstration  of 
the  first  inequality  in  (3.15),  we  must  show  that  H >c^(N)I 
where  c^(N)>0  and  l/c^(N)  is  bounded  algebraically  as 

Since  H..  is  nearly  a constant-diagonal  tridiagonal  matrix, 

eigenvalues  of  H can  be  studied  by  standard  techniques:  if 

D„  = det  (II., -XI)  , then  D.,  satisfies  the  three-term  recurrence 
N N N 

relation 


<Vjk 


c j 

-%c. 

-’sc. 


Dn  = (1-X ) D, 


N-l 


1 

4 


D. 


N-2 


(N>2) 


(8.17) 


the 
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Since  (8.17)  has  constant  coefficients,  it  is  ensv  to  solve 

exactly.  From  this  solution,  it  is  not  hard  to  show  that  the 

smallest  eigenvalue  of  H satisfies 

N 


> (N)  ^ 1,2 
nun  8n2 


(N->*’)  . 


Choosing  c^  (N)  = Xmin(N)  gives  1/c^  (N)  . BN2/ it 2 (N>«)  . 

This  proves  that  the  norm  defined  by  (8.14)  is  algebraically 
equivalent  to  the  Chebyshev  norm  and,  therefore,  Chcby shev-Galerkin 
approximation  to  (8.1)  is  algebraically  stable.  Note  also  that  (8.13) 
shows  that  the  matrix  11^  defined  in  (8.1G)  satisfies  (5.7b)  with 
c(N)  = 0.  Since  ||HN||  = 0(1)  and  J J n"1 J J * 0(N2),  (5.11) 

implies  that  | | exp(L^t) | | = 0(N)  as  N>>«,  which  also  follows 
directly  from  (8.15). 

We  have  not  yet  beer,  able  to  obtain  a rigorous  demonstration 
that  | | exp (L^t) { | = 0 (N1//4)  as  N as  found  numerically  in 
Table  8.2.  Our  best  result  to  date  is  ||exp(L^t)||  = 0(N)  as 


Although  the  problem  (8.1)  is  not  well  posed  in  the  Chebyshev  norm 
(as  shown  in  Sec.  5),  it  is  well  posed  in  the  norm  defined  by  (8.14). 
Using  (8.1)  and  (8.3),  we  obtain 


B BWMM 
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L, 


. Vi  \l/2  o 

A ^(m)  u 


so  that  ||eLt||<l  in  the  norm  (8.14). 

Proof  of  Algebraic  Stability  for  Chcbyshov-Tau  Approximation 

The  proof  of  algebraic  stability  for  the  tau  method  is  similar 
to  that  just  given  for  Galerkin  approximation.  The  Chebyshev-tau 
approximation  satisfies 


8UN  9UN 

at  + 3x  TN(t)  tn^x) 


uN(-l,t)  = 0 » 


(8.18) 


where 


N 


’Jo*"  T" 


(8.19) 
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~ TJT:^  - . 


I 


Therefore 


3 u 

<!-*>  3>T9 1 


daM  r 

N __N  N T + } b T 

~ N dt  N n n 


N-l 

l 

n=0 


(8.20) 


Moreover,  comparing  the  coefficients  of  x on  both  sides  of 
(8.18)  we  find 

da 


N 


TN(t)  = “dt 


(8.21) 


Eqs . (8.18-21)  imply 


32u 


32u, 


da 


N,  2 


,8u»  ,,  ° V + (1_x)UX  , . in  ( 8 22) 

(TT'U  (TF' (1  x,3x3t’  2 dt 


Since 


3un 


at  x =-i 


= o 


we  obtain 


^ 2 
a u. 


2 (— -^-  (1-x)  --  ) 

1 at  ' u x>  dxdt  ' 


j (i-x)1/2(i+x)  1/2aouN/at)2/ax  dx 


-l 


= / (i-x)"1/2(i+x)"3/2(auN/at.) 2 dx. 


Therefore  / (8. 21)  gives 


d r1  ,,  v ,,  2,-i  ,3uN.  2 , < . 
g-£  / (1-x)  (1-x  ) (-^-)  dx  - 0 


-1 


(8.23) 
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This  proves  that  the  evolution  of 
(8.14).  Finally,  the  boundedness  of 


^UN 

-r — is  stable  xn  the  norm 
auN 

— — implies  the 

O X 

boundedness  of  uvl,  as  will  now  be  shown.  If  u , is  given 

N N 

by  (8.1*)),  then 


% 

3x 


N-l 


“ I 


n=0 


T 

n 


where 


an  = 


c . b , - b , , 
n-l  n-l  n+1 

2n 


(n  = 1 , . . . ,N) 


The  boundary  condition  uN(-l,t)  = 0 requires  that 

N 

a0  = ^ an 
n=l 

Therefore,  since  N is  bounded,  so  is  iv. 

■sir  N 

In  Sec.  9 we  present  a variety  of  numerical  results  for 
the  numerical  solution  of  (8.1)  by  Chebyshev  and  Legendre  spectral 
methods . 

Effect  of  Boundary  Conditions  on  the  S t a bill t y_ of  Spectra)  Me  thod s 

Let  us  discuss  the  effect  of  boundary  conditions  on  the 

stability  of  the  Cheybshev  approximations  to  (8.1).  In  Sec.  6 it 

was  shown  that  incorrect  treatment  of  the  boundary  does  not  affect 

the  stability  (though  it  docs  affect  the  convergence)  of  the  Fouricr- 

Galerkin  method.  This  is  not  the  case  for  the  Chobyshev-spectral 

methods.  Let  us  assume  that  we  solve  (8.1)  ignoring  the  boundary 

condition  (8.3)  and  suppose  that  u (x,0)  = T (x).  The  resulting 

N N 


system  of  Galerkin  equations  for  (an)  is 


3 

= - | I pa  (8.24) 

~ cn  p=n+l  p 
p+n  odd 

where  an(0)  = 6 . Eq.  (8.24)  can  easily  be  solved:  aN_k(t) 

is  a polynomial  in  t of  degree  k of  the  form 

aN_k(t)  = (-2 ) k (k)  tk  + ...  . (8.25) 

This  solution  is  clearly  not  bounded  by  any  finite  power  of  N. 

Thus,  the  Chebyshev  methods  are  algebraically  unstable  when  no 
boundary  conditions  are  applied. 

If  we  had  imposed  the  boundary  condition  u(+l,t)  = 0 in 
addition  to,  or  instead  of,  the  boundary  condition  u(-l,t)  = 0, 
then  Chebyshev-spectral  solution  to  (8.1)  would  be  unstable. 

With  u(+l,t)=0  instead  of  (8.3),  the  Chebyshev-spectral  approximations 
to  the  operator  -3/3x  all  have  eigenvalues  with  positive  real  parts 
(that  grow  as  N + ®) . Similarly,  if  we  tried  to  impose  the  extra 
boundary  condition  3u(+l,t)/3x  = 0 in  addition  to  u(-l,t)=0  [as 
is  frequently  done  with  finite  difference  methods] , an  unstable 
scheme  would  result. 

The  effect  of  imposing  u(+l,t)  = 0 in  addition  to  u(-l,t)  = 0 
is  slightly  different  for  Leqendre-spectral  methods.  With  u(-l,t)= 
u(+l,t)=0,  Legendre-spectral  methods  for  solution  of  (8.1)  are 
semi-bounded.  In  fact, 

* r1 

(v, (L+L  )v)  = -2  / v3v/3x  dx  = 0 

-1 

when  v(±l,t)  = 0,  so  these  methods  are  semi-bounded  and  stable. 


LJk 
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However,  these  spectral  approximations  are  not  consistent. 
For  example,  Galerkin  approximation  involves  expansion  of 


u(x,t)  in  terms  of  the  functions 


>2n  (x)~p2n (x)  ~ P0(1) 


♦ 2n+i  pi  <x>  that  satisfy  ( ± 1)  = 0. 


2n+l 


But  3u/3x  cannot,  in  general,  be  expanded  in  terms  of  the 
functions  <(>^(x)  • 

The  above  situations  are  typical  of  rapidly  converging 
spectral  methods.  Spectral  methods  are  extremely  sensitive  to 
the  proper  formulation  of  boundary  conditions.  When  proper 
boundary  conditions  are  imposed  so  the  problem  is  well  posed, 
the  methods  yield  very  accurate  results;  when  improper  boundary 
conditions  are  mistakenly  applied,  the  methods  are  likely  to  be 
explosively  unstable.  The  stability  and  convergence  of  spectral 
methods  follows  very  closely  that  of  the  exact  equations. 
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cj 


l. 


c 


c 


9.  Time  Differencing 

In  previous  sections  we  have  investigated  the  properties  of 
spectral  approximations  to  the  spatial  operator  L of  the 
differential  equation 


In  this  section  we  investigate  the  properties  of  time-integration 
techniques  for  the  solution  of  the  semi-discrete  spectral  approx- 
imations 


3uN 

3t  LN°N 


(9.1) 


Time  discretization  errors  in  both  finite  difference  and 
spectral  methods  are  typically  much  smaller  than  are  spatial 
discretization  errors.  There  are  two  reasons  for  this:  (i)  time 
steps  are  frequently  restricted  in  size  by  explicit  stability 
conditions  — stability  of  the  time  integration  requires  that 
time-differencing  errors  be  small;  and  (ii)  many  problems  involve 
several  space  coordinates  so  any  possible  efficiency  in  the 
representation  of  the  spatial  variation  of  the  dependent  variables 
is  quite  important  to  the  overall  efficiency  of  the  method — if 
the  number  of  degrees  of  freedom  necessary  to  describe  a certain 
three-dimensional  field  accurately  can  be  reduced  by  two  in  each 
space  direction  then  the  total  number  of  degrees  of  freedom  is 
decreased  by  a factor  8,  but  a similar  improvement  in  time 
differencing  gives  just  a factor  2.  We  will  investigate 
here  only  finite-difference  methods  of  finite-order  accuracy  for 
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timewise  solution  of  (9.1)  despite  the  infinite-order 
accuracy  in  space  of  many  of  the  spectral  methods  discussed 
in  earlier  sections.  No  efficient,  infinite-order  accurate 
time-differencing  methods  for  variable  coefficient  problems 
are  yet  known.  The  current  state-of-the-art  of  time-integration 
techniques  for  spectral  methods  is  far  from  satisfactory  on  both 
theoretical  ^nd  practical  grounds  and  the  results  to  be  presented 
here  must  be  regarded  as  only  3 beginning. 

One  of  our  prime  goals  is  to  investigate  the  stability  of 
time  differencing  methods  for  the  solution  of  (9.1  ).  To  do 
this  we  must  first  explain  how  to  extend  the  stability  definitions 
given  in  Sects.  4 and  5.  Let  u^(x)  = u^^nAt)  be  the  approx- 
imation to  the  solution  of  (10.1)  at  time  nAt,  where  At  is  a 
time  step.  Time  differencing  methods  involve  approximating 


in  some  way  to  give  a rule  for  constructing  u 


n+1 

N 


n+1  „ ... . n 

UN  - V4t,UN  , 


(9.2) 


where  is  an  operator  acting  on  uN- 

it  follows  that 


Using  this  rule  repetitively 


u^(x,nAt)  = [KN(At) ]nuN(x, 0) , 


(9.3) 


where  for  notational  simplicity  we  assume  At  fixed.  We  say  that 
(10.2)  is  strongly  stable  if 


| | [1^  (At)  ]n|  | < K (nAt) 


(9.4) 


L 


for  all  N and  n sufficiently  large  and  At  sufficiently  small. 
Here  R(T)  is  a finite  function  of  T.  We  define  generalized 
stability  by  replacing  K(T)  in  (9.3)  by  n^-+^K(T)  as  in(5»2) 

A sufficient,  though  not  necessary,  condition  for  strong 
stability  (9.4  ) is  > • 


IV  At) 


-1  < icAt 


(9.5) 


for  some  finite  k and  all  At  sufficiently  small.  If  K^At) 
is  a normal  matrix  then  stability  is  assured  if  the  eigenvalues 
X of  Kjj  satisfy  the  von  Neumann  condition 


maxi  X I < 1 + xAt 


(9.6) 


i 


for  sufficiently  small  At  (Richtmyer  & Morton  1967) . If 
is  not  normal,  then  (9.6)  is  still  a necessary,  though  not 
sufficient,  condition  for  stability  in  the  sense  of  (9.4  ). 


The  importance  of  these  stability  definitions  is  that  they 
lead  to  the  fully  discrete  form  of  the  equivalence  theorem  (see 
Sec.  4):  a scheme  is  consistent  if 


K(At)  - I 

I I ( N ' At - L)u|  I - 0 


(9.7) 


as  N + « and  At  -*•  0 for  all  u in  a dense  subspace  of  H; 
scheme  is  convergent  if 

{ | u ” - u(nAt)  | | -*■  0 

as  N -*•  » and  At  -*■  0 for  all  n satisfying  0 <_  nAt  < T and 
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all  u(0)  e^f.  The  equivalence  theorem  states  that  for  consistant 
approximations  to  well-posed  problems,  stability  is  equivalent 
to  convergence. 

Let  us  now  study  the  stability  properties  of  some 
specific  time-differencing  methods. 

Implicit  time-integration  methods 


Two  time-integration  methods  that  are  unconditionally 
stable  for  every  algebraically  stable  spectral  method  are  the 
Crank-Nicolson  scheme  and  the  backwards  Euler  scheme.  For  any 
semi-discrete  spectral  approximation  (9.1)  to  ufc  = Lu,  the 
Crank-Nicolson  time-differencing  scheme  is  given  by 


n+1  n 

n+1  n . L T , UN  +UN  , 

UN  - UN  = “ V 2 1 


and  the  backwards  Euler  scheme  is  given  by 


(9.8) 


n * ■ t n n+1 

“n  - 4t  ln  us 


(9.9) 


To  prove  that  (9.8)  or  (9.9)  is  stable,  we  proceed  as  follows 
If  (9.1)  is  algebraically  stable  there  exists  a family  of 
positive  definite  Hermitian  matrices  (HN>  such  that 


hn  ln  + V hn  i “(N)  % 


or,  equivalently. 


H..1/2  L_.  H ”1/2  + H “1/2  L * H„1/2  < a (N)  I , 


where  a(N)<d£nN  for  some  finite  d.  Substituting 


v n = H u n 
N N N 


into  (9.8-9),  we  obtain,  respectively, 


where 


n+1.  n-1 

v , +v  ‘ 

_ n+1  ..  n . ^ „ # N N ^ 

VN  ' VN  " At  % ( 2 } ' 


VS+1  -VNn  = AtMNVNn+1' 


1/2  H -1/2 
mn  " hn  ln  hn 


Taking  the  scalar  product  of  ( 9 .10)  with 
VN  + VN+1/  we  get 


(9.10) 


(9.11) 


iivN"+in2-iivlinii2  = f «vNnti  v ■ > 


N N 


llV+1+vNn||2<  S|£  [||vNn+1||2+||vNn||2I 


(9.12) 


.J 
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w . ----- 


Therefore , 


l|v"+ln25  IH-  >»  oAt) 

N (1-  J5  a At) 


iivHnii2. 


(9.13) 


which  proves  generalized  stability  for  vN  and,  hence,  also  for 


„ -1/2 

UH  * HN  VN- 


Similarly,  we  may  show  that  the  backwards  Euler  method 

is  unconditionally  stable.  Taking  the  scalar  product  of  (9.11) 

...  n+1  , n ..  . 

with  vN  + vN  , we  obtain 


i n+1 1 1 2 n,  | 2 . , ...  n+-l  n n+1. 

^ VN  ^ ^ ^VN  A VN  ' V N V N 


. . n+1  n+1  . . „ n+1. 

= it  vN  , 2vn  - i vN  ) 


< ||vNn+1||2 


(9.14) 
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so  that 


’ ^ 


iiV+1nJiiiitiiv»nii2  • 


proving  generalized  stability  of  u^. 

Note  that  the  above  proofs  show  that  if  n(N)  is  not  a 
function  of  N then  v»,  = H„1,/2  u„  is  strongly  stable  for  both 

N N N 

the  Crank-  Nicolson  and  backwards  Euler  schemes. 

Spectral  approximations  using  Fourier  series 

Next,  we  consider  several  time  integration 
methods  for  Fourier  series  spectral  approximations  to 

ut  + “x  * 0 

with  periodic  boundary  conditions.  As  shown  in  Sec.  6,  the 
collocation  equations  are 


= C DCu. 


19.15) 


where  the  matrices  2Nx2N  C and  D are  defined  in  (6.3). 

The  'leapfrog'  time  differencing  approximation  to  (9.15)  is 
the  explicit  two-level  scheme 


n+1  n-1  -1  n 

UN  * UN  = 2At  C °CUN 


(9.16) 


in  the  leapfrog  scheme 

- ujj-1  + 2AtC-1DCuJJ  , 
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so  Kjj  is  a two-level  evolution  operator  since  it  depends  on 

both  un_1  and  u?..  The  definitions  of  stability,  convergence, 

N N 

and  consistency  given  above  extend  easily  to  this  case. 

We  shall  show  that  (9.16)  is  strongly  stable  provided  that 


At  < 


(9.17) 


I 


To  show  this  we  first  recall  from  Sec.  6 that  C is  unitary 
and  D is  skew-Hermitian.  Therefore,  A = C ^DC  is  also  skew- 
Hermitian,  and  hence  normal,  so  that 
| | A | | = 2tt  (N-l)  . 

Now  we  take  the  inner  product  of  (9.16)  with  uP+*  + un_1 

N N 

to  get 


n+1 i , 2 


, n-l  i | 2 _ . . „ , n+1  , n-l  . n. 

K,  II  = 2AtRe(uM  + uM  , Au  ) , 


since  u5!~^"  and  u[l  are  real.  Since  A*  = 
N N 


-A,  we  obtain 


Un  = ||ur 

M I 1 IS 


.n,  i 2 


n+1  . n. 


+ | | u“ M “ - 2AtRe (u“  , Au“) 


“n'I2  - l l "S’1 l l 


. n . n-l . _ n-l 

- 2AtRe(uN,  Aun  ) = UN 


50  US  = 4 


Schwarz'  inequality  implies  that 


Re(uJJ+1,AujJ)  < | | A | | | |i 
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so  that  if  (9.17)  is  satisfied,  i.e.  At|  f A | | < 1-e  for  some  t > 0, 
|24tRe(u£+1,  AuJJ)|  < 2 (1-e)  ||u£+1||  Hujll  . 

Using  this  result,  j^e  obtain 

G<l|ujJ+1||  + ||uJJ||2)  + (1-e)  (|!ujj+1||  - Mujjl  | 2 ) < uJJ  » U° 


(9.18) 

Since  uN  is  a bounded  function  of  N (because  of  the  initial 
conditions),  we  see  that  )|uJJ+1||  is  bounded  for  all  N and 
n , proving  strong  stability. 

Another  way  to  prove  that  the  leapfrog  and  Crank-Nicolson  time 
differencing  schemes  are  strongly  stable  for  (9.15)  is  to  use  a 
modal  analysis,  which  is  justified  because  A is  normal.  Thus, 
if  uf.  is  an  eigenfunction  of  A with  eigenvalue  A , the 
Crank-Nicolson  approximation  to  KN(At)  is 

•ft 

K„(At)u?  = (1  + 4 A At ) / ( 1 - ~ XAt)  u°  (9.19) 

N N ^ £ N 


Since  the  eigenvalues  A of  C ^DC  are  all  pure  imaginary,  it 
follows  that  ||KN(At)||  * 1,  so  Crank-Nicolson  differencing  is 

stable . 

Still  another  time  differencing  method  for  solution  of  (9.15) 
is  to  use  a Runge-Kutta  scheme.  It  easily  verified  the  first  and 
second-order  Runge-Kutta  methods  are  unstable  unless  At  satisfies 
conditions  that  are  much  more  restrictive  than  (9.17).  With  the 

first-order  Euler  method 


u 


n+1 

N 


. , . n 
+ AtAu^  , 
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stability  requires  that  N At  be  bounded  as  At  -+■  0 [because 
I 1 (At)  J J = 1 + 0(N2  At2)]  ; with  the  second-order  scheme 


G"+1/2  = u"  + i AtAu" 


N 2 


n+l  n ..  ~n+l/2 
UN  = UN  + AtAUN 


stability  requires  that  N 7 At  be  bounded  as  At  -*•  0 . However, 
the  third  and  fourth-order  Runge-Kutta  methods  give  conditional 
stability  restrictions  like  (9.17)  which  we  will  now  derive. 

The  third-order  Runge-Kutta  scheme  may  be  written  f of  a linear 
equation  like  (9.1)  as 


[I  + AtA  + 1/2  ( AtA)  2 + 1/6  ( AtA)  3 ) uf!  = K„  (At)u" 

N N N 


(9.20) 


Since  (At)  given  by  (9. 20)  is  normal, 


Kjg  ( At) 


= max  |1  + AAt  + l/2(XAt)2  + l/6(AAt)3j 


where  the  maximum  is  taken  over  all  the  eigenvalues  of  A. 
eigenvalues  of  A are  ik  with  |k|  < 2tt(N-1),  so  (9.6  ) is 
satisfied  provided  that 


At  < 


. /T 

2 it  (N-1) 


(9.21) 
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Thus,  this  method  allows  time  steps  that  can  be  /T~  times 
larger  than  with  the  leapfrog  scheme  while  maintaining  stability. 
However,  if  the  operator  A is  complicated,  the  third-order 
Runge-Kutta  scheme  requires  about  3 times  as  much  work  as  leap- 
from  at  each  time  efcep,  so  it  is  probably  not  competitive. 

Similar  analysis  of  the  fourth-order  Runge-Kutta  scheme 
gives  the  stability  condition 


At  < 


ST 

tr  (N-l) 


(9.22  ) 


Thus  time  steps  can  be  nearly  three  times  larger  than  with 
leapfrog  steps.  However,  fourth -order  Runge-Kutta  differencing 
requires  about  four  times  the  work  of  leapfrog  differencing,  so 
the  scheme  is  probably  not  too  useful  unless  very  high  accuracy 
is  desired. 

Now  we  shall  consider  time-differencing  methods  for  Fourier 
series  spectral  approximations  to  the  heat  equation  with  periodic 
boundary  conditions: 


u.  = u (0  < x < 1) 

t XX  - - 


(9.2  3) 


Collocation  using  Fourier  series  gives  the  spectral  equations 


IT  = c_1d2cun 


(9.24  ) 
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The  matrix  C DC  is  negative  definite.  Because  (9.19)  still 
holds  and  all  eigenvalues  X are  negative,  Crank-Nicolson  time 
differencing  is  unconditionally  stable.  On  the  other  hand,  it 

is  easy  to  show  that  leapfrog  differencing  is  unconditionally 

0 -12. 
unstable.  In  fact,  if  uN  is  an  eigenfunction  of  C DC  with 

eigenvalue  X < 0 then  | | ( At) nu^ | | grows  like 

(-X At  + /+(XAt)  2)  ~ e-\(nAt)  as  At  o for  fixed  X and 

2 2 2 

nAt.  Since  max|X|  = 4 it  (N-l)  grows  like  N as  N 
| | ( At ) nu^ | | cannot  be  bounded  be  a finite  function  of  nAt 
for  all  N,  proving  unconditional  instability. 

Another  way  to  solve  (9.24)  is  to  use  a generalized  Dufort- 
Frankel  scheme 


n+1 


n-l 

“n 


~2aF 


C_1D2Cu"  - 
N 


.,2.  n+1 

YN  (un 


- 2 


“N 


n-l , 
UN  > 


(9.25) 

2 

If  y > n then  this  method  is  unconditionally  stable 
(Gottlieb  & Gustaffson  1976). 

Similarly,  Euler  time  differencing  of  (9.24)  is  conditionally 

2 

stable.  Stability  requires  that  At  max|X|  <_  2 or  At  i'2”*'(N-l)  ] 
Higher-order  Adams-Bashf orth  schemes  have  similar  conditional 
stability  limits. 


Time-differencing  for  mixed  initial-boundary  value  problems 

Some  care  is  necessary  in  the  formulation  of  time- 
differencing  methods  for  spectral  approximations  to  mixed 
initial-boundary  value  problems.  The  sensitivity  of  spectral 
methods  to  the  proper  formulation  of  boundary  conditions, 
as  shown  in  Sects.  6-8,  carries  over  to  the  formulation 
of  time-differencing  methods  for  these  approximations.  For 
example,  for  most  mixed  initial-boundary  value  problems  leap- 
frog time  differencing  is  unconditionally  unstable  for  spectral 
approximations.  Furthermore,  explicit  time  integration  methods 
may  be  unduly  restricted  by  conditional  stability  requirements 
in  spectral  approximations.  The  origin  of  these  severe 
restrictions  is  the  very  high  resolution  of  spectral  methods 
near  boundaries.  Thus,  it  is  frequently  necessary  to  combine 
special  kinds  of  implicit  time- integration  methods  with  spectral 
approximations  in  order  to  maintain  high  accuracy  at  reasonable 
computational  cost.  Several  examples  will  be  given  later. 

Let  us  beqin  by  studying  time-differencing  methods  for 
the  Chebyshev-spectral  approxima t ion  to  the  mixed  initial- 
boundary  value  problem  (8.1-3); 

ut  ♦ ux  » 0 (-l^_x^l,  t>0)  , (9.27) 

u (x, 0)  * f (x)  (-l'xvl) , (9.28) 


u (-1 , t ) = 0 (t>0 ) . 
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(9.29) 


In  Sec.  8,  we  proved  that  various  semi-discrete  spectral 
approximations  to  (9.27-29)  are  algebraically  stable. 

Let  us  first  consider  the  leapfrog  time-differencing 
scheme 


“N 


n+1 


1 


+ 2At  L 


n 

N “n  ' 


(9.30) 


where  u?,(x)  is  the  time-discretized  approximation  to 
N 

uN(x,nAt),  At  is  the  time  step,  and  the  semi-discrete 
approximation  is  3u^/3t  * Ljj  u^. 

This  scheme  is  unconditionally  unstable  for  any  At  as  N-*® . 

To  show  this  we  recall  that  in  Sec.  8.  we  proved  that  the 

eigenvalues  of  L^  have  negative  real  part  (see  Table  8.3) 

and  that  the  largest  eigenvalue  of  L„  has  a neqative  real 

N 

2 

part  that  grows  like  N"  as  N*®.  Let  us  rewrite  (9.30) 
in  the  2x2  block-matr ix  form 


/UN*‘\  /2“L  N ^ 


X N 


0/ 


n-1 


(9.  U) 


If  the  eigenvalues  of  LN  are  denoted  as  uN,  then  the 
eigenvalues  of  the  matrix  on  the  right  in  9.31  are 


X 


(t) 

N 


UN  at 


X / 1 + 


~2  T 
(At,2  UN2 


(9.32) 


-182- 


For  fixed  N and  At-»-0, 


XJ7  « e (1  + 0(At2)) 


Thus 


(9.33) 


f"]‘ 


n 

(“D  e ( 1 + 0 (At) ) (0<nAt<T,  Af*-0) 


(9.34) 


Since  | | Kn (A t) n| | >( | n and  there  are  eigenvalues  of 
Ln  with  negative  real  part  of  order  N , no  inequality  of 
the  form  (9.4)  can  be  satisfied.  Thus,  leapfrog  time 
differencing  of  the  Chebyshev  approximations  to  (9.27-29) 
is  unconditionally  unstable. 

There  are  several  conditionally  stable  explicit  time- 
differencing  approximations  that  can  be  used  with  spectral 
approximations  to  (9.27-29).  Two  examples  are  the  Adams- 
Bashforth  scheme 


n-fl  n ^ 3 ..  Tn  1 T n-1 

“n  un  + 5 At  ln  " 2 At  ln 


and  the  modified  Euler  scheme 


(9.35) 


"S+1  » uj  ♦ At  Ln  u£ 


US+1  * US + i At  ln  + ln  ^+1 


(9.36a) 


(9.36b) 
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The  modified  Euler  scheme  (9.36) is  in  practice  stable  provided 
the  stability  condition 


8 


At  < — «- 
JT 


(9.37) 


is  satisfied.  A similar  stability  condition  holds  for  the 
Adams-Bashforth  scheme. 

The  fact  that  the  stability  limit  in  (9.37)  depends 
2 

on  1/N  rather  than  1/N  is  not  very  surprising 

. % 

because  the  Chebyshev  collocation  points  cos  irn/N 

2 

are  spaced  by  a distance  of  order  1/N  near  the  boundaries. 

Since  the  wave  speed  in  (9.27)  is  1 the  wave  propagates  from 

2 

one  grid  point  to  the  next  in  a time  of  order  1/N  so  time 

steps  must  be  smaller  than  this  to  maintain  explicit  stability. 

The  explicit  stability  restriction  (9.37)  for  Chebyshev- 

spectral  methods  with  N polynomials  should  be  contrasted  with 

the  corresponding  stability  conditions  for  finite  difference 

approximatiorB  to  (9.27-29).  with  N gridpoints  uniformly  spaced 

in  the  interval  -1<x<_1,  the  grid  spacing  is  2/N  so  the 
Courant  stability  condition  is  At  <_  2/N.  As  N-*-®,  this 

stability  condition  on  finite  difference  schemes  is  much 

weaker  than  the  condition  (9.37)  on  the  spectral  approximations. 

A semi-implicit  technique  that  permits  stable  time-differencing 

with  spectral  methods  with  a stability  condition  like  that 

of  finite-difference  schemes  will  be  discussed  later  in  this 

section. 


-184- 


IM  . 'll  li  I pp«| 


mmmmmm 


In  order  to  prove  that  the  modified  Euler  method  (9.36) 
is  stable,  we  begin  by  noting  that  (9.36)  is  equivalent  to  the 
second-order  Taylor  series  method 


uN  1 - (I  + At  Ljj  + 2(At)  ^ ^ ~ GN  uN  (9.38 

A sufficient  condition  for  algebraic  stability  of  (9.38)  is 
the  existence  of  positive-definite  symmetric  matrices  SN 
such  that 


T 

G S G < S 
N N N - N 


(9.39a) 


and  the  condition  number  of  S„  satisfies 

N 


I lsNl  I I ISjj1!  | ■ 0(Ne)  (N-*-°°)  . 


(9.39b) 


for  some  finite  6 . if  (9.39)  holds  then 


"$>***  <Vni  ^n_1  SN  (G^-1  < ...  < SN 


or 


Therefore, 


(gn) 


n 


c-1/2 

SN 


< 


I. 


f Is 


1/2 

N 


< 1, 
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so  that 


MuSlI  = IKS)n  uSMillsN1/2IIMsJ/2(GN)n  sn1/2N 

i isn1/2i  i 1 1 “Si i = 0 (nBiiuSm>  (n^- 


To  complete  the  stability  proof  we  must  investigate 
under  what  conditions  matrices  SN  satisfying  (9.39)  exist, 
One  choice  for  S,  is  just  the  Liapounov  matrices  of  L ; 
these  matrices  satisfy 


SN  LN  + LN  SN  1 


(9.40) 


It  was  shown  in  Sec. 8 that  the  Liapounov  matrices  for  spectral 
approximations  to  (9 . 27-29)  have  algebraically  bounded  condition 
number.  Using  (9.38)  , we  obtain 

Gj  Sn  Gn  - [I  + At  tjj  + | (At)2(L^|T]  SN[l+AtLN4<lt>2lLN)^ 


or 


GS  SN  S * SN  + 4t  <LN  SN  + SN  V 


- t |<*t>2  [(L2)1  S„  * 21*  S„  L„  t S,  1 

+ 7 <At)3t(L2)T  SN  LN  + Lj  Sn  L2I  + i (At)4(L2)T  S„  L2 
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From  (9.40),  it  follows  that 


(ln)T  sn  + sh  ln 


T 2 

L..  SM  L + S_T  L . 
N N N d N 


,T  2,  T „ t . T T _ T 2 
<V  SN  LN  LN  SN  LN 


T 

"ln  ln 


so  that 


SN  GN  = SN  - 4tI  - <At)2  [LN  + V 


- 7 l4t>3  ln  ln  + T <At>4  (ln>T  sn 

Thus  , (9.39a)  is  satisfied  provided  that 


■At  (L  + L ) < 1 
N N — 


At  Lm  Sm  L < 2 I 
N N N — 


(9.41) 


(9.4  2) 


If  (9.41-42)  are  satisfied  then  the  modified  Euler  method  foi 
(9.27-29)  is  algebraically  stable. 

At  first  , it  may  appear  that  the  stability  condition 
(9.42)  is  much  more  severe  than  the  stability  condition 
(9.4} ).  In  fact,  we  know  from  Sec.  8 that 


I |LN  | | = 0 (N2 ) , | |SNI  | « 0(1)  (N-»)  , 


-187- 


4 

so  that  (9.42)  seems  to  require  that  At=0(l/N  ) as 
However,  the  stability  condition  (9.42)  is  no 
more  restrictive  than  the  stability  condition  (9.41) 
To  see  this  we  use  (9.40)  written  in  the  form 


LN  SN  ln 


+ 


u^r1 


r 


to  obtain  the  representation  [see  (5.13)] 


T 00  . _ . 

LN  SN  LN  = / expl(LN  ) t]  exp [L^  t]  dt  . 

0 (9.43) 

It  may  be  shown  that  the  norm  of  the  integrand  of  (9.43) 

2 

is  0(1)  as  N ->  ® for  t = 0(N  ) and  that  the  norm  decays 
rapidly  to  zero  as  t -*■  Therefore, 


llLN  SN  Ln  M = 0 <N2)  (»*”>  W.44) 

showing  that  the  stability  condition  (9.42)  is  of  the  form 
At  = 0(1/N2). 
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semi- implicit  methods 

When  explicit  time-stepping  methods  are  used  to  solve 
semi-discrete  spectral  equations  for  the  hyperbolic  problem 

+ a (x)  |^  = 0 (-1<X<1)  (9.45) 

d t dX 


with  appropriate  boundary  conditions  [that  depend  on  the  sign 
of  a(x)],  there  result  stability  conditions  of  the  form 


At  < min  { 


1 

N2|  a (1)  | 


HmaxTaTxri  * <9-46’ 

I*I<1 


These  stability  limits  can  be  derived  heur istically  from  the 
Courant  stability  condition 


At  < 

<** 


Ax 

Ja 


eff 


eff 1 


(9.47) 


where  aeff  is  the  effective  wave  propagation  speed  in  a 

direction  in  which  there  is  effective  grid  resolution  Axgff. 

Near  the  boundaries  x=±l,  the  Chebyshev-spectral  methods  have 

resolution  ^xeff  = Od/N2)  as  N-*-oo  while  aeff  = a(±l)» 

the  interior  of  -1<x<1,  Chebyshev  series  have  effective  resolution 

Axeff  * 0 < H 1 as  N-t-oo  while  the  largest  wave  spped  is  max|a(x)|. 

Thus,  (9.47)  implies  (9.46)  for  the  Chebyshev-spectral  methods. 

The  stability  condition  (9.46)  is  too  severe  for  many 

2 

applications  because  it  requires  that  At  = 0 (1/N  ). 


In  order  to  relax  this  severe  constraint,  we  use  a semi-implicit 
method  in  which  the  propagation  through  the  high-resolution  boundary 
is  treated  implicitly,  but  the  propagation  through  the  interior  is 
treated  explicitly. 

One  possible  semi-implicit  scheme  is  the  following  two-step 
ntethod  . Let  Ln  be  the  Chebyshev-spectral  approximation  to 
-a  (x)  3—  with  appropriate  boundary  conditions  applied,  and 

, L~  be  the  Chebyshev  spectral  approximations  to  the 
constant  coefficient  wave  operators  - a(+l)3/3x,  -a(-l)3/3x, 

respectively,  again  with  appropriate  boundary  conditions  applied. 

A semi-implicit  two-step  scheme  is  given  by 

- ***  LN  - »S  + f»t  <LN  - V <9-48a) 

“T1  - «■;  c1  - “S4  + fi*  <ln  - ln>  uS4  i9-,8b> 

The  scheme  (9.48)  is  stable  if  the  stability  condition 


At  < 


1 

N max  | a (x)  | 


(9.49) 


is  satisfied. 

The  condition  (9.49)  is  sufficient 
but  the  semi- implicit  scheme  (9.48)  may 
(9.49)  is  violated.  If  max|a(x)|<|  a ( 1 ) | 


to  ensure  stability, 
be  stable  even  if 

or  max|  a (x)|  < | a (-1)|  , 
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(9.48)  is  usually  unconditionally  stable  for  sufficiently 
large  N (see  Sec.  8 of  Orszag  1974) . The  implementation 
of  (9.48)  on  a computer  is  straightforward  and  efficient;  the 
properties  of  Chebyshev  polynomials  summarized  in  the  Appendix 
show  that  the  implicit  equations  (9.48)  are  essentially  tridiagonal 
matrix  equations. 

The  reason  that  the  semi-implicit  method  outlined  above 

2 

does  not  have  a stability  restriction  like  At  = 0(1/N  ) can  be 
understood  as  follows.  By  subtracting  L+  and  L~  in  succeeding 
half-time-steps,  the  explicit  part  of  the  calculation  is  similar 


to  that  in  solving  an  equation  of  the  form 


3u  . ,,  2.  . 3u 

Ft  + (1~x  } b(x)  ITx 


(9.50) 


where  the  wave  speed  vanishes  at  x*±l.  If  b(x)  = b,  a 
constant,  the  Chebyshev- tau  equations  for  (9.50)  are  just 


dF=  2 ^b  I (n-1)  a|n_1(  - (n+1)  an+1  ] 


(9.51) 


where  c,  = 2 and  c = 1 for  n>0.  By  Gerschgorin ' s theorem. 
On 

I I Ljg I I for  (9.51)  satisfies 


| |Ln|  | = 0(bN)  (N-*-a>)  , 


(9.52) 


so  the  explicit  time  step  restriction  is  At  = 0(l/bN)  as 
N-*®. 
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We  note  that  Chebyshev-spectral  approximations  to  (9.50) 
are  stable  when  no  boundary  conditions  are  applied.  In  fact, 
using  Galerkin  approximation  and  the  Chebyshev  inner  product, 
we  obtain 


9u 


(u 


N' 


N 

at 


+ b(l-x  ) 


% , 

3x 


0 . 


so 


1 U, 


N 


-9-  [ 

dt  -1  /1-x* 


dx  = -b  / /1-x2  uN  dx 


i 2 , „ 2 

1 XUM  1 UM 

-b  / -==  dx  < |b|  / -J==  dx 

-1  /1-x2  -1  /1-x2 


Therefore, 

I |uN(t) | |2  < e|b|t||uN(0)||2  . 


proving  stability. 

There  are  other  attractive  semi-implicit  schemes  for  (9.45). 

For  example,  suppose  a(x)  is  one-signed,  say  a(x)>0,  and  let 

a * maxa(x)  . Define  L™aX  as  the  Chebyshev  approximation  to 
max  n 

- ^ amax  with  boundary  conditions  imposed  at  x = -1 . A 

semi-implicit  Chebyshev  spectral  scheme  for  (9.45)  is 

n+l  A.  .max  n+1  „n  , A.  . max . n ,Q 

UN  _At  LN  N * “N  + At(LN  LN  > UN  * (9*53) 
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The  scheme  (9.53)  is  usually  unconditionally  stable  and  avoids 
the  severe  time  step  restriction  (9.46).  It  is  also  easy  to 
implement  efficiently  because  L™ax  is  a Chebyshev  approximation  to 
a constant-coefficient  wave  operator.^ 

The  same  kind  of  trick  stabilizes  spectral  methods  for  non- 
linear equations.  For  example,  if  we  are  solving  the  equation 


JLU  + 
3t 


u 


jiu 

ax 


o 


during  a time  interval  in  which  u(x,t)  is  smooth  (no  shock  waves 
then  we  may  use  the  semi-implicit  scheme 


■2H  + lu 

gt  2 max 


au 

ax 


l 

7 


u 

max 


u) 


ax 


in  which  the  terms  on  the  left  are  treated  implicitly  in  time, 
while  those  on  the  right  are  treated  explicitly.  Here  u is 

an  estimate  of  the  largest  value  of  u(x,t).  Similar  semi-implicit, 
methods  are  effective  in  eliminating  (or  at  least  relaxing)  time- 
step  restrictions  for  finite-difference  methods.  The  key  idea 
is  to  recognize  the  source  term  of  a numerical  instability  and  then 
to  approximate  it  by  a simple  expression  that  can  easily  be  treated 
implicitly. 


D.  Haidvogel  has 
mu 


(9.53)  with  L 


max 
N 


pointed  out  that  the  semi— implicit  scheme 
. . _ .„1N  replaced  by  a Chebyshev  spectral  approximation 

to  (bx+c)  3/3x,  where  b+c  = a(+l),  c-b  = a(-l),  is  also 
stable  under  the  weak  restriction  (9.49).  The  resulting 
implicit  equations  are  still  tridiagonal  [see  (A. 9) t (A.18)]. 
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Several  other  examples  of  semi-impl icit  methods  should  make 
the  general  technique  clear.  For  the  variable  coefficient  heat 
equation 


Ut  = k(x)  Uxx 


with  suitable  boundary  conditions  at  x -i  1 and  k(x)>0, 
Chebyshev-spectral  methods  give  explicit  time-step  stability 
conditions  of  the  form 


At  < min  { : 


k ( - 1 ) N 


k (1)  N 


n max  k (x) 

M<i 


(9.54) 


The  very  severe  time  step  restriction  that  At:  = 0(1/N  ) as 

N-*"  is  due  to  the  high  resolution  of  Chobyshev  series  near  the 

boundaries  x = ± 1 . To  avoid  this  problem  we  can  use  a semi-implicit 

method.  Let  L , be  the  Chebyshev-spectral  approximation  to 
N 

k(x)3^/3x^  and  let  L™nx  be  the  Chebyshev-spectral  approximation 

N 

12  2 ... 
to  — k 3 /3x  where  k = maxk(x)  . The  semi-implicit 

2 max  "'nv 


max 


scheme  (9.53)  with  L™ax  defined  in  this  way 

seems  to  be  unconditionally  stable  (Orszag  1974)  and  certainly 
does  not  have  any  stability  restrictions  of  the  form  (9.54). 

Finally,  we  comment  on  the  need  for  implicit  or  semi-implicit 
schemes  in  multi-dimensional  pi'oblems.  If  wo  wish  to  solve  the 
Navier-Stokes  equations 


-► 

3u  . hj  ► 

ft  + U • V u 


= - Vp  + W u 


(9.55) 


V .u 


= 0 
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for  incompressible  fluid  flow,  the  treatment  of  the  various 
terms  should  guided  closely  by  the  type  of  stability  restrictions 
they  impose. 

If  v-  0 then  we  need  only  consider  the  types  of  stability 
restrictions  induced  by  the  advective  term  -u.Vu  and  by  the 
pressure  term  -$p;  we  will  not  discuss  the  effect  of  the 
pressure  because  it  is  closely  connected  to  the  incompressiblity 
condition  V.u»0  and  is  not  relevant  to  the  semi-implicit  ideas 
discussed  here.  At  a boundary  of  the  flow,  it  is  appropriate  to 
specify  boundary  conditions  on  u.n  where  n is  the  normal 
to  the  boundary.  If  the  boundary  is  solid  and  stationary,  then 
u.n-0  and  we  are  in  a situation  similar  to  that  modelled  by 
(9.50).  The  effective  convective  speed  normal  to  the  boundary 
vanishes,  so  spectral  methods  exhibit  no  unusual  time  stepping 
restrictions.  However,  if  fluid  is  being  blown  into  or  sucked 
out  of  the  boundary  so  u.n^O,  then  semi-implicit  methods  must 
be  applied  to  avoid  unreasonably  restrictive  conditions  like 
(9.46)  on  the  time  steps. 

If  v>0,  then  the  viscous  terms  in  the  Navier-Stokes  equations 
should  be  treated  implicitly  to  avoid  unreasonable  time  step 
restrictions  due  to  the  high  resolution  of  spectral  approximations 
near  the  boundary. 


10 . Efficient  Implementation  of  Spectral  Methods 

There  are  two  aspects  of  the  efficient  implementation  of 
spectral  methods  that  we  discuss  here:  (1)  evaluation  of 
derivatives;  (ii)  evaluation  of  nonlinear  and  nonconstant 
coefficient  terms;  (iii)  roundoff  errors.  More  details  on 
these  matters  are  given  elsewhere  (see  the  References) . 


Evaluation  of  derivatives 


An  efficient  procedure  to  obtain  the  expansion  coefficients 
of  derivatives  of  a function  f(x)  in  terms  of  the  expansion 
coefficients  of  f(x)  is  to  use  recurrence  relations.  For 
example,  to  evaluate  the  term 

N 

sn  i , paP 

p*n+l 

p+n  odd 

that  appears  in  the  Chebyshev  equations  (2.11),  (2.19),  and 
(2. 32), we  use  the  recurrence 


Sn+2  + (n+1)an+l 


(0<n<N-l) 


(10.1) 


with  S.  ■ * 0.  In  this  way,  S„  is  evaluated  for  all  n 

N N't!  n 

using  only  N arithmetic  operations.  The  existence  of  the  recurrence 
relation  (10.1)  is  ensured  by  the  recurrence  property 


T ’ 

Tn+1 


T ' 
n-l 


(n>l) 
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satisfied  by  the  Chebyshev  polynomials.  Similarly,  it  is  possible 

to  derive  recurrence  relations  to  evaluate  efficiently  the 
coefficients  of  arbitrary  derivatives  of  functions  expanded  in 
Chebyshev  and  other  classical  polynomial  expansions. 

Evaluation  of  nonlinear  and  nonconstant  coefficeint  terms 
The  most  efficient  way  to  evaluate  nonlinear  and  general 
nonconstant  terms  in  spectral  approximations  is  to  apply  transform 
methods.  The  key  idea  is  to  apply  fast  Fourier  transforms  and  other 
transforms  to  transform  efficiently  between  spectral  representations 
of  a function  f(x)  and  physical-space  representations  of  f (x)  . 

With  Chebyshev  polynomial  expansions,  fast  Fourier  transforms  permit 
the  evaluation  of  arbitrary  nonlinear  and  nonconstant  coefficients 
terms  in  order  N log  N arithmetic  operations. 

In  general,  collocation  methods  give  approximations  to  nonlinear 
and  nonconstant  coefficient  problems  that  can  be  more  efficiently 
implemented  than  Galerkin  or  tau  approximations.  Collocation  is 
recommended  for  these  problems.  For  example,  the  solution  of  the 
hyperbolic  problem 

ft  + eU+X  fx  = <-1  i x 1 1#  t > 0)  , (10.2) 

u(-l,t)  = 0, 

would  be  difficult  using  Galerkin  or  tau  approximation  but  is 
straightforward  using  collocation  methods. 


Let  us  explain  how  to  march  the 
by  one  time  step  efficiently  using 
the  N+l  collocation  points  x • = 
and  represent  the  current  solution 


solution  to  (10.2)  forward 

Chebyshev  collocation.  We  introdu 
cos  Tij/N  (j  = 0,...,N) 

u . as 
3 


u 


3 


N 

Z 

n-0 


a 


n 


cos 


iinj 

N 


(10.3) 


Then  we  invert  (10.3)  by  the  fast  Fourier  transform  to 

obtain  a for  n = 0,1,..., N and  calculate 
n 


' 2Sn/c 


n 


by  (10.1).  Next  we  evaluate 


?u 

8x 


X=X  3 


N 

Z 

n=0 


cos 


irni 

N 


(10.4) 


using  the  fast  Fourier  transform.  Finally,  we  evaluate 
exp(Uj+x^) (Du/3x) j at  each  of  the  'grid'  points  x^ 

and  use  the  results  to  march  the  solution  forward  to  the  next 
time  step. 

For  quadrat ically  nonlinear  differential  equations,  like  the 
Navier-Stokes  equations  of  incompressible  fluid  dynamics,  Galerkin 
and  tau  approximations  are  workable  but  normally  requii e at  least 


twice  the  computational  work  of  collocation  approximation.  However, 
Galerkin  approximation  is  sometimes  very  attractive  because  it  gives 
approximations  that  are  conservative  and  have  no  so-called  aliasing 
errors  (see  Orszag  1971c,  1972  for  a more  complete  discussion  of 
these  properties) . 

Roundoff  Errors 

Transform  methods  normally  give  no  appreciable  amplification 
of  roundoff  errors.  In  fact,  the  evaluation  of  convolution-like 
sums  using  fast  Fourier  transforms  often  gives  results  with  much 
smaller  roundoff  error  than  would  be  obtained  if  the  convolution 
sums  were  evaluated  directly. 

On  the  other  hand,  the  use  of  recurrence  relations  to  evaluate 
derivatives  can  sometimes  be  a source  of  large  roundoff  errors. 

In  this  case  , it  is  often  best  to  convert  the  problem  being  solved 
into  a new  one  that  is  numerically  well-conditioned.  An  example  of 
such  a transformation  is  given  below. 

Example  10.1:  Solution  of  y"-ky=f(x)  by  Chebyshev  polynomials 

The  boundary-value  problem 


y"  - ky  = f(x)  -1<x<1 


(10.2) 


y (-1)  » A,  y (1)  = B 

can  be  solved  using  a Chebyshev-tau  approximation.  The 
resulting  approximation  yN(x)  is  given  by  (see  Appendix) 


'V*1  ’ J0  % Vx) 


(10.3) 
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— I p(p2-n2)  a - ka  = f (0<nfN-2)  (10.4) 

cn  p=n+2  P n n 

p+n  even 

N N 

l (-1)“  a = A,  l a = B,  (10.5) 

n=0  n n=0  n 


where  {f  } are  the  Chebyshev  series  coefficients  of  f(x) 


n 


The  solution  of  the  system  (10.4-5)  for  the  Chebyshev 
coefficients  (an>  may  be  done  in  several  ways.  One  obvious 
way  to  do  this  efficiently  is  to  write 


n 


= aiX)  + aai2)  + 0aj3)  . 


Here  a 


(1) 


n 


satisfies  a 


N 


'n  r _n 

(1)  = a(1!  = 0 and 
N N-l 


(10.6) 


— I p(p2-n2)a(1)  -ka(1)  =f  (0<n<N-2), 
Cn  p=n+2  P n " ~ " 


n 


while  a 


(2) 


n 


satisfies  a 


(2) 

N 


(2) 

= 1'  aN_i  = 0 and 


N 


--  I p (p2-n2) a 


(2)  (2) 


- ka 


and  a 


(3) 


n 


'n  p=n+  2 


satisfies  a 


n 


= 0 (0<n<N-2) , 


(3) 

N 


= 0, 


(3)  . . 

aN_l=  and 


p(p2-n2)a(3) - ka(3)  = 0 (0<n<N-2) . 

p+n  even 

Each  of  the  solutions  a^3^  » a^2^  ' an^  ' ma^  ^ound 

using  roughly  N operations  by  backwards  recurrence.  When 
the  constants  a and  6 in  (10.6)  are  chosen  so  that  the 
boundary  conditions  (10.5)  are  satisfied,  an  given  by 
(10.6)  satisfies  (10.4-5). 

The  above  procedure  is  efficient  but  it  is  not  usually 
numerically  stable.  Roundoff  errors  multiply  rapidly  so  that 


1 N 
— I 

Cn  p=n+2 
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¥ 


a^  may  have  little  significance. 

A better  procedure  is  to  first  convert  (10.4-5)  into  a 
nearly  tridiagonal  system  of  equations.  It  may  be  shown  that 
(10.4-5)  is  equivalent  to  the  system 


kc 


n-2 


ke 


4n (n-1) 


n 


-2 


- (1  + 


n+2 

F 


) a 


2 (n-1) 


n 


*en+4  s 
4n (n+l)  n+2 


= cn-2fn-2 
4n(n-l) 


en+2^n  . en+4^n+2 

I^T)  ^TSTnTIT 


(2<n<N) 


(10.7) 


with  the  boundary  conditions  (10.5)  still  applied.  Here 
c =2 , c =1  for  n>0  and  e =1  for  n<N,  e=0  for  n>N.  The 
system  (10.5),  (10.7)  may  be  solved  by  standard  banded  matrix 
techniques  in  roughly  the  number  of  operations  required  to 
solve  pentadiagonal  systems  of  equations.  The  equations  in  the 
form  (10.7)  are  essentially  diagonally  dominant  so  no  appreciable 

accumulation  of  roundoff  errors  occurs.  This  technique  for 
solution  of  (10.2)  is  very  useful  in  implementing  implicit 
spectral  methods  for  dissipative  terms  and  for  solving  Poisson- 
like  equations  (see  Sec.  14). 
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11.  Numerical  Results  for  Hyperbolic  Problems 


We  begin  by  presenting  numerical  results  for  spectral  approximations 
to  the  problem 

u + u - 0 (-l<x<l,t>0)  (11.1) 

t x 

u(x,0)  * 0 ,u(-l,t)  - g(t),  (11.2) 

whose  exact  solution  is 

i g (t  - x - 1)  (x<t-l) 

u(x,t)  * { 

(0  (x>t-l).  (11.3) 

If  g(t)  is  smooth,  u(x,t)  is  smooth  for  |x|<l  when  t>2?  when 
t<2,  u(x,t)  is  not  smooth  at  x*t-l . 

In  Sec.  2 wc  explained  how  to  obtain  semi-discrete  Galerkin, 
tau,  and  collocation  approximation  to  (11.1-2)  using  either 
Chebyshev  or  Legendre  polynomial  expansions.  In  Sec.  9,  we  showed 
that  either  Adams-Bashforth  or  modified  Euler  time  differencing  gives 
stable  and  convergent  results  for  these  spectral  approximations.  The 
numerical  results  cited  in  this  Section  were  obtained  by  Adams-Bashforth 
time-differencing;  time  steps  were  chosen  small  enough  that  time- 
differencing  errors  are  negligible. 

Comparison  of  Chebyshev  and  Legendre  Polynomial  Spectral  Methods  for 
Smooth  Solutions 

When  g(t)  — sinMut,  the  solution  (11.3)  has  M complete 
waves  within  | x | <^  1 when  t>2.  As  argued  in  Sec.  3,  we  expect  that 
accurate  results  will  be  obtained  only  if  N^Mit  polynomials  are 
retained. 
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In  Fig.  11.1,  we  plot  the  root-mean-square  error  for  |x|<_l 
averaged  in  time  for  4^t£4.4  obtained  using  the  Chebyshev 
approximations  to  (1 1.1-2)  when  g(t)  = -sin  5irt  . In  this  time 
interval,  u(x,t)  is  smooth  for  |x|<l.  Observe  that  the  errors 
decrease  exponentially  fast  when  N>5tr.  Also  observe  that  when  the 
spectral  approximations  are  relatively  inaccurate  (errors  greater 
than  roughly  10%) , Galerkin  approximation  is  most  accurate  followed 
by  collocation  and  then  tau.  On  the  other  hand,  when  the  spectral 
approximations  are  very  accurate  (errors  less  than  roughly  0.5%), 
tau  approximation  is  most  accurate  followed  by  Galerkin  and 
collocation.  This  behavior  seems  typical.  Also  observe  from 
Fig.  11.1  that  all  three  spectral  approximations  are  nearly  as 
accurate  as  the  best  (rms)  Chebyshev  approximation;  in  fact,  tau 
approximation  with  N+l  polynomials  is  usually  more  accurate  than 
the  best  approximation  with  N polynomials.  Here  the  best  (rms) 

Chebyshev  approximation  is  that  Nth  degree  polynomial  that 

r1  i 1 2 . . 2. -1/2. 

minimizes  / | — u | (1-x  ) dx. 


In  Fig.  11.2,  we  make  similar  comparisons  of  the  error  in 
spectral  approximations  using  Legendre  series  for  the  problem 
(10.1-2)  with  g (t)  = -sin  5"t  . Here  too  the  errors  decrease 
exponentially  fast  when  N>5tt.  Again,  tau  approximation  is  more 
accurate  than  Galerkin  when  both  are  very  accurate,  while  it  is 
less  accurate  when  both  are  relatively  inaccurate.  Also,  tau 
approximation  with  N+l  polynomials  and  Galerkin  approximations 
with  N+2  polynomials  are  more  accurate  than  the  best  Legendre 
approximation  with  N polynomials.  Here  the  best  Legendre 

approximation  is  that  Nth  degree  polynomial  that  minimizes 

1 2 
/ K “ u|  dx. 

-1 
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Fig.  11.1.  A plot  of  the  Lj-errors  in  Chebyshev-spectral 
solution  of  (11.1-2)  with  g(t)  - -sin  5irt.  The  errors 
are  averaged  in  time  over  the  interval  4<t<4.4;  the  exact 
solution  u(x,t)  = sin  5ir  (x+l-t)  is  smootK  throughout  this 
time  interval.  The  best  (rms)  approximation  is  given  by 
(3.41)  with  M « 5,  a * 1-t  trunc'ated  after  T„(x).  Observe 
that  the  errors  decrease  rapidly  for  N > 5tr. 
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Fig.  11.2.  Same  as  Fig.  11.1  except  for  Legendre-spectral 
solution  of  (11.1-2)  with  g(t)  ■ -sin  5irt.  Here  the  best 
(rms)  approximation  is  given  by  (3.45)  with  M = 5,  a = 1-t 
truncated  after  PN(x). 
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In  Fig.  11.3-4  we  plot  the  error  eN(x,t)  in  the  best 

Chebyshev  polynomial  approximation  to  sin  5 *(x+l-t)  at  t=4 . 

Observe  that  eN(x,t)  is  nearly  an  'ecIual  ripple'  approximation 

(Acton  1970)  so  u„(x,t)  is  nearly  a minimax  approximation. 

In  Figs.  11.5-8  we  plot  the  errors  eN(x,t)  versus  x 

at  t*4  obtained  by  numerical  solution  of  Chebyshev  spectral 

approximations  to  (11.1-2).  As  N increases,  the  tau  method 

gives  the  closest  approximation  to  an  equal-ripple  error,  which 

is  consistent  with  the  result  shown  in  Fig.  11.1  that  tau  approximation 

gives  the  smallest  errors  at  high  accuracy. 

In  Figs.  11.9-10,  we  plot  the  error  in  the  best 

Legendre  polynomial  approximation  to  sin  5tt(x  +l-t)  at  t=4 . 

Observe  that  e^(x,t)  has  large  errors  near  the  boundaries 

x = ±1.  By  comparing  the  results  plotted  in  Figs.  11.3-4  with 

those  plotted  in  Figs.  11.9-10,  we  conclude  that  the  best  Chebyshev 

polynomial  approximation  is  closer  to  an  equal  ripple  approximation 

than  is  the  best  Legendre  polynomial  approximation.  Even  though  the 

best  Legendre  polynomial  approximation  to  u(x,t)  gives  the  smallest 

mean-square  error  to  u,  the  best  Chebyshev  polynomial  approximation 

usually  gives  a smaller  value  of  the  maximum  pointwise  (Lro)  error. 

The  large  errors  of  the  best  Legendre  approximation  are  concentrated 

near  the  boundaries  x*±l,  while  the  Chebyshev  weight  function 
2 -1/2 

( 1— x ) tends  to  distribute  the  errors  in  the  best  Chebyshev 

approximation  uniformly  throughout  -1<x<1 . 
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Chebyshev-Collocat 


Fig.  11.10  Same  as  Fig.  11.9  except 


In  Figs.  11.11-13,  we  plot  the  errors  eN(x,t)  at  t«4 
obtained  by  numerical  solution  of  Legendre  spectral  approximations 
to  (11.1-2).  As  for  Chebyshev-spectral  approximations,  the  error 
in  Legendre- tau  approximation  is  smaller  than  that  in  Legendre- 
Galerkin  approximation. 

One  important  feature  of  Legendre-spectral  approximation  is 

that  the  spatial  distribution  of  the  error  in  tau  and  Galerkin 
approximation  plotted  in  Figs.  11.11-13  differs  markedly  from 

the  spatial  distribution  of  the  error  in  the  best  Legendre 
polynomial  approximations  plotted  in  Figs.  11.9-10.  The  boundary 
errors  in  the  best  L2  approximation  are  relatively  large  while 
the  boundary  errors  are  relatively  smaller  in  the  spectral 
approximations . 

The  boundary  (endpoint)  errors  in  Legendre-tau  approximation 
exhibit  ' superconvergence ' in  the  sense  that  they  go  to  zero  much 
faster  than  either  the  L2  - errors  or  the  L2  and  endpoint  errors 
of  Chebyshev-tau  approximation.  This  fact  is  illustrated  in  Fig. 11. 14 
where  we  plot  the  L2  and  endpoint  errors  of  Legendre-tau  and 
Chebyshev-tau  spectral  approximations  to  the  solution  of  (11.1-2) 
with  g(t)  =-sin  5nt  . Here  the  endpoint  error  is  | uN(+l  ,t) -u(+l  ,t)| 
at  the  outflow  boundary  x*=  +1. 

Several  features  of  the  results  plotted  in  Fig.  11.14  deserve 
comment.  First,  although  the  maximum  error  of  the  best  N-term 
Chebyshev  polynomial  approximation  is  smaller  than  the  maximum 
error  of  the  best  Legendre  polynomial  approximation  to  u(x,t)  by 
roughly  a factor  1//N  [see  (3.38)  and  (3.39)],  the  maximum  error 
of  the  Legendre-tau  approximation  is  smaller  than  the  maximum  error 
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Fiq.  11.14  A comparison  of  the  Chebyshev-tau  and  Legendre- 
tau  L2  and  endpoint  (x  = -*-1)  errors  for  the  solution  to  (11.1-2) 
with  g(t)  = -sin  5nt. 
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of  the  Chebyshev-tau  approximation.  Second,  the  endpoint  error  at 
x=l  of  the  Legendre-tau  approximation  goes  to  zero  like  the  square 
of  the  endpoint  error  of  the  Chebyshev-tau  approximation.  This 
remarkable  behavior  of  endpoint  errors  in  Legendre-polynomial 
approximations  was  found  origdnally  by  Lanczos  in  a slightly 
different  context  [Lanczos  1966  (p.  156),  1973]. 

A mathematical  analysis  of  the  errors  of  spectral  approximations 
to  (11.1-2)  has  been  given  recently  by  Dubiner  (1977).  Dubiner's 
results  include:  (a)  asymptotic  estimates  of  the  errors  incurred 
by  the  various  spectral  methods,  including  error  oscillations  when 
the  solution  is  smooth;  (b)  a complete  boundary  layer  description 
of  the  decay  of  large  errors  due  to  discontinuities  after  the 
discontinuities  propagate  out  of  the  computational  domain;  (c) 
analysis  of  the  behavior  of  the  tau-f unction  t (t)  in  (2.34).  Dubiner 
has  analyzed  a variety  of  spectral  methods  for  (11.1-2)  based  on 
expansions  in  general  Jacobi  polynomials . His  ingenious  analyses 
of  tau  methods  should  permit  more  complete  analysis  of  these 
methods  than  possible  using  earlier  a posteriori  analysis  (see  Fox 
& Parker  1968  for  examples  of  a posteriori  error  analysis  of  tau 
methods) . 

Mesh  Refinement 

Sometimes  it  is  useful  to  split  up  a domain  into  several 
subdomains  and  then  use  numerical  methods  of  different  spatial 
resolution  in  each.  For  example,  in  limited-area  numerical  weather 
forecasting  near  a metropolitan  area,  it  may  be  desirable  to  have 
much  finer  resolution  in  a small  region  than  is  practical  globally. 

One  way  to  do  this  is  to  solve  the  problem  separately  on  each 


* 
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of  several  subdomains  and  then  to  match  the  numerical  solutions 
so  obtained  across  subdomain  boundaries.  As  a model  of  this 
procedure  we  consider  the  problem 


u + u =0  (-1<x<1»  t>0) 

t x 

u(-l,t)  * g(t) , 
v.  + v =0  (l£X£3,t>0) 

v(l+,t)  = u(l-,t) . 

With  finite  difference  methods,  the  accurate  solution  of  the  coupled 
system  (11.4.5)  using  different  grids  for  -l^x^l  than  for 
l<xf_3  may  be  troublesome.  Inaccurate  results  or  even  numerical 
instabilities  can  result  from  the  matching  (Browning,  Kreiss  & Oliger 
1973)  . Because  grids  with  different  grid  separations  have  different 
dispersion  characteristics  for  waves  propagating  on  the  grid, 
waves  can  reflect  from  the  boundary  at  x=l  and  cause  large 
errors. 

Spectral  methods  are  attractive  for  the  solution  of  mesh 
refinement  problems  like  (11.4-5)  because  they  give  small  endpoint 
errors.  For  example,  the  Chebyshev-tau  approximation  to  (11.4-5) 
with  N+l  polynomials  to  represent  the  solution  for  -1<^x<^1  and 
M+l  polynomials  for  l<_x<_3  is  given  by 

N 

uN(x,t)  * £ an ( t)  Tn (x)  (-lixil)  (11.6) 


(11.5a) 

(11.5b) 


(11.4a) 

(11.4b) 


-2?  I- 


(l<x<  3) 


(11.7) 


M 


vM(x't)  " L bm(t)  VX'2) 

m=U 


dt 


2 N 

- — j pa 

cn  p=n+l  P 

p+n  odd 


( 0<n<N-l) 


(11.8) 


db 
m 

dt 


N 


- - zr  l 


cm  p=m+l 
p+m  odd 


P 


(0<m<M-l) 


(11.9) 


l (-l)n  a - g(t) 
n=0  n 


(11.10) 


Mm  N 

l <-D  b - I a 

m | n 

m= 0 n=  u 


(li.iD 


It  may  easily  be  shown  that  if  g(t)  is  smooth,  the  solution  to 

(11.6-11)  converges  to  the  solution  of  (11.4-5)  throughout 

-l<x<3  faster  than  any  finite  power  1/N  or  1/M  as  N,  M-*00. 

The  solutions  for  an<^  ^<.x<_3  match  without  the 

necessity  of  imposing  any  matching  conditions  in  addition  to 

(11.5b).  Because  no  spurious  downstream  boundary  conditions 

are  applied  at  x=+l  on  the  wave  propagating  in  the  interval 

-1<x<1,  there  are  no  reflected  waves. 
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One  more  example  of  a refined  mesh  spectral  calculation 


is  instructive.  Consider  the  heat  equation  problem 


yt  v 


-l<x<  1 


(11.12a) 


= vl_v 

9t  \x2 


l<x<  3 


u(-l,t)  = f(t),  v ( 3, t) =g (t) 


u(l-,t)  = v ( l+,t),  |H  (l-,t)=  |X  (l+,t) 


(11.12b) 


(11.12c) 


(Il.l2d) 


where  (ll.l2d)  follows  by  requiring  continuity  of  temperature  and 
heat  flux  across  the  boundary  at  x*l.  A Chebyshev-tau  approximation 
to  (11.12)  is  given  by 


u(x,t)  = l an<t)Tn(x) 
n*0 


(-1<x<1) 


(11.13a) 


v(x,t)  - J b ( t) T (x-2 ) 

W*0  m m 


(l<x<  3) 


(11.13b) 
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da 

n 

-ar 


(11.13c) 


db 


m 

~St 


N j 5 

-2-  l p(p  -iT)  a (0<n<N-2) 

Cn  p=n+2  P 

p+n  even 

M 

^ p(p2-m2)b  (0vp<M-2) 

c P=m+2  H * P 

p+m  odd 


l (-l)na  = f(t),  l b - g(t) 
n=0  n m=0 


(11.13d) 


(11 . 13e) 


N 

l 


n=0 


a 

n 


M 

l 


m=0 


( - 1 ) mb  , 
m 


IN  9 

l n2a 
n=0  n 


M 


l (-1) mm2b 


m=0 


m 

(ll.l3f) 


It  may  be  shown  as  in  Example  7.1 (v)  that  this  approximation  is 
semi-bounded  and  hence  stable  and  convergent. 

Discontinuities 

When  t<2,  the  solution  (11.3)  to  (11.1-2)  is  not 
smooth  at  x»t-l ; if  g(t)  = sinMnt  , the  solution  has  a 
discontinuous  derivative.  This  discontinuity  seriously  degrades 
the  rate  of  convergence  of  spectral  approximations  near  the 
discontinuity.  Nevertheless,  spectral  approximations  are  still 
normally  much  more  accurate  than  finite-difference  approximations 
to  the  same  problem.  Orszag  & Jayne  (1974)  give  comparisons 
between  finite-difference  and  spectral  approximations  to 
discontinuous  solutions;  in  particular,  they  argue  that  if  the 
gth  derivative  of  the  solution  is  discontinuous,  the  rate  of 
convergence  of  Chebyshev-spectral  approximations  to  (11.1-3)  for 
t«-2  is  of  order  l/N^  as  N-*a>.  Dubiner  (1977)  has  given  a detailed 
asymptotic  analysis  of  this  problem.  His  results  include  detailed 
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behavior  of  the  error  for  all  x and  t and  are  in  good  agreement 
with  numerical  solutions. 

One  of  the  attractive  features  of  spectral  methods  for  problems 
with  discontinuities  is  that  the  region  of  large  errors  is  more 
localized  near  the  discontinuity  than  in  finite-difference  methods. 
Thus,  it  should  be  possible  to  eliminate  oscillations  near  the 
discontinuity  using  less  dissipation  than  is  required  when  finite 
difference  methods  are  used.  A comparison  of  the  error  in  Chebyehev- 
tau  and  second  and  fourth-order  solutions  of  (11.1-2)  for  t<2  is 
given  in  Fig.  11.15. 

Another  interesting  way  to  use  spectral  methods  for  problems 
with  discontinuous  solutions  has  been  suggested  by  Boris  & Book 
(1976) . The  "optimal  flux-corrected  transport"  approximation 
gives  good  resolution  of  discontinuities  without  introduction  of 
unphysical  numerical  oscillations  near  the  discontinuity.  The  idea 
is  to  add  in  an  artificial  diffusion  to  smooth  the  discontinuity  and 
then  to  'anti-diffuse'  the  resulting  solution  in  such  a way  that  no 
new  oscillations  or  maxima/minima  are  produced. 

Comparison  with  Finite  Difference  Methods 

Finite-difference  approximations  to  (11.1-2)  must  be 
formulated  carefully  near  the  boundaries  x =±  1.  For  example, 
the  fourth-order  semi-discrete  approximation 


8 •frfl~uj-l) ~uj  + 2+Uj-2 
12A  x 


0 


where  u^(t)  = u(jAx,t) , must  be  modified  at  x=-l+Ax,  1-Ax,l 
because  u(-l-Ax,t) , u*  +Ax,t) , u(l  + 2Ax,t)  all  lie  outside  the 
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computational  domain  -l<x^l . Kreiss  & Oliger  (1973)  discuss 

methods  to  formulate  difference  approximations  at  these  grid 

points.  However,  it  is  not  known  how  to  formulate  appropriate 

'boundary*  conditions  for  arbitrary  order  difference  schemes. 

This  difficulty  is  an  artifact  of  difference  methods;  a fourth- 

order  difference  quation  requires  4'boundary'  conditions  while 

only  1 condition  (11.2)  is  properly  imposed  on  the  first-order 
IJ 

differential  equation  (11.1). 

On  the  other  hand,  properly  formulated  spectral  methods 
require  no  'spurrous'  boundary  conditions.  Indeed,  the  imposition 

11 

of  a spurious  boundary  condition  on  a spectral  approximation  to 
(11.1),  like  3u/3x  =0  at  x = +1 , will  induce  an  unconditional 
instability  (see  Sects.  8,12).  The  mathematics  of  spectral 

<: 

approximations  follows  closely  the  mathematics  of  the  differential 
equation  being  solved. 

Spectral  approximations  also  require  considerably  fewer  degrees 
of  freedom  to  achieve  accurate  results  than  are  required  by 
difference  methods.  A comparison  for  the  problem  (11.1-2)  is 
given  in  Table  11.1  for  late  times  at  which  the  solution  is  smooth. 

In  Figs.  11.16-19  we  show  three-dimensional  perspective  plots 
of  the  solution  to  a simple  two-dimensional  hyperbolic  problem  with 
periodic  boundary  conditions 


3Aix,y, t) 
Tt 


„ 3A (x,y,t)  3A(x,y,t) 

y 3x  x 3y 


0 


(11.14) 


<; 
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Table  11.1 


Second-order 

N M 

1 

G2 

Fourth-order 

N M 

G4 

N 

Chebyshev- 

M 

-tau 

e 

00 

40 

2 

0.1 

20 

2 

0.04 

16 

4 

0.08 

80 

2 

0.03 

30 

2 

0.008 

20 

4 

0.001 

160 

2 

0.008 

40 

2 

0.002 

28 

8 

0.2 

40 

4 

1. 

40 

4 

0.07 

32 

8 

0.008 

80 

4 

0.2 

80 

4 

0.005 

42 

12 

0.2 

160 

4 

0.06 

160 

4 

0.0003 

46 

12 

0.02 

Table  11.1.  l2  (rms)  errors  for  the  solution  of  (11.1-2)  with 
g(t)  = sinMTrt.  The  errors  listed  are  measured  at  t=5  when  the 
solution  (11.3)  is  smooth.  Time  differencing  errors  are  negligible 
and  N is  the  number  of  grid  points  or  Chebyshev  polynomials. 

Observe  that  to  achieve  a 1%  error,  the  second-order  method  requires 
N/M>40,  the  fourth-order  method  requires  N/m>15,  while  the 
Chebyshev-tau  method  requires  N/M^tt  . 
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with 


A ( x±  2n  , y ± 2tt  , t ) = A (x,y , t) 


The  solution  to  (11.14)  is  constant  along  the  characteristics 


x+iy=  (xQ  + iy0)ext.  Therefore,  A(x,y,2u)=  A(x,y,0)  so 
the  solution  should  keep  A unchanged  after  a time  2n . In 


Fig.  11.1 6,  we  plot  the  initial  conditions  used  for  the  calculation 


whose  results  are  plotted  in  Figs.  11.17-19.  In  Fig.  11.17  we  plot 


the  results  at  t=2ir  of  a second-order  centered  space  difference 


scheme;  in  Fig.  11.18  we  plot  the  results  of  a fourth-order  scheme 


and  in  Fig.  11.19  we  plot  the  results  of  a spectral  calculation 


using  the  Fourier  expansion 


A(x,y,t)  = l l a(k,p,t)eikx+ipy 

| k | <K  |p|<P 


All  three  calculations  used  the  same  number  of  degrees  of  freedom 


but  the  differences  in  accuracy  are  striking.  The  Fourier-spectral 


method  works  well  even  though  the  convecting  velocity  (-y,x)  in 


(11.14)  has  jump  discontinuities  at  x=*2tt,  y=±2ir . The  dominant 


error  in  all  three  calculations  originates  from  the  'corners'  of 


the  initial  A(x,y,0)  distribution;  thus  error  appears  as  a large 


lagging  phase  error  in  the  finite  difference  solutions  which  explanins 
the  'wakes'  of  large  errors  following  the  remnants  of  A(x,y,2it). 


Higher-Order  Wave  Equations 


The  mixed  initial-boundary  value 


-2  2 

= A-H  (-l<x<l,t>0) 

at  ax2 


(11.15) 
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Fig.  11.16  a perspective  plot  of  the  A(x,y,t=0)  used  in 
numerical  test  of  methods  to  solve  (11.14). 
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--- 


Fig.  11.18  Sane  as  Fiq.  11.17  except  using  a fourth-order 
centered  difference  scheme  on  a 64  x 64  grid. 


-r 


\ 


U(±lft)  = 0 (11.16) 

u(x, 0)  = f ( x ) , (x,  0)  = g (x)  (11.17) 

is  well  posed.  Legendre  polynomial  solution  of  (11.15-17)  is 
semi-bounded  and,  hence  stable  (see  Sec.  7) . However,  we  have  not 
yet  been  able  to  prove  that  Chebyshev  solution  of  this  problem  is 
ever  algebraically  stable.  The  techniques  of  Sec.  8 prove  that 
if  the  boundary  conditions  (11.16)  are  replaced  by  the  characteristic 
conditions 

9u(-l,t)  + 3u (-1 , t)  = 0/  3u(l, t)  _ 9u(l,t)  _ 0> 

9t  9 t '9t  9t 

the  scheme  is  algebraically  stable.  However,  we  have  not  yet 
been  able  to  prove  this  result  for  (11.16).  However,  it  is 
reassuring  to  note  that  we  have  solved  the  Cheybshev-spectral 
approximations  to  (11.15-17)  and  find  no  evidence  of  lack  of 
convergence.  Indeed,  the  Chebyshev  methods  work  just  as  well  as 
they  do  for  (11.1-2).  Thus,  it  is  not  the  spectral  methods  that 
run  into  difficulty  on  higher-order  equations,  but  just  our 
methods  of  analysis. 
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12.  Advective-Dif fusion  Equation 


In  this  section,  we  consider  spectral  methods  for  the 
advective-dif fusion  ('linearized  Burgers')  equation 


3u(x, t)  A „ 3u(x,t) 
Jt  u 3x 


2 

v + f (x , t ) 

3x* 


(-1  < x < 1)  (12.1) 


u(-l.  t) 


0 , u(l,t) 


(12.2) 


u (x,  0)  « g (x) 


(12.3) 


Eq.  (12.1)  is  parabolic  so  boundary  conditions  should  be  applied 
at  both  x = -1  and  x * +1  . When  v is  small,  the  boundary 
condition  applied  at  x = +1  (assuming  U > 0)  has  an  interesting 
effect  on  the  stability  of  the  spectral  methods. 

To  begin,  we  remark  that  the  analyses  of  Sects.  7-8  can  be 
extended  to  show  that,  as  N °°,  N-term  Legendre  and  Chebyshev 
approximations  to  (12.1-3)  are  stable  and  convergent. 

For  example,  Chebyshev-Galerkin  approximation  is  stable 


because  (12.1-2)  and  (7.3)  imply  that 


d 

It 


f1  — - — dx  < |u|  f °2  -372  dx  - v f 

]-l  j-l  (l-x2) 3/2  J-1  (l-x 


7^72 


dx 


|-/l-v/u  u2 

J-/l'-v7u  (l-x2) 3//i 


dx 


< ef  f1  -a 
■ u J-i  r. 


dx 


(12.4) 


£2 


so  the  approximation 


is  semi-bounded. 
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However,  for  finite  N,  there  may  be  difficulty  integrating 

the  resulting  spectral  equations.  With  Legendre  polynomials, 

* 

Galerkin  approximation  to  (12.1-3)  satisfies  <,  0 

so  there  is  no  difficulty  with  time  integrations  (although  the 
solution  may  not  be  accurate  unless  N is  large  enough. 

On  the  other  hand,  Chebyshev-spectral  solution  of  (12.1-3) 
encounters  the  following  curious  behavior  when  v is  small. 

If  v/u  is  small  and  N is  not  too  large,  the  Chebyshev-spectral 
approximations  LN  to  (12.1-3)  have  eigenvalues  with  positive 
real  parts.  In  Table  12.1,  we  list  values  of  Ncrit  for  various 
values  of  v/U;  for  N < Ncrit  , for  Chebyshev-tau 

approximation  to  (12.1-3)  has  eigenvalues  with  positive  real 

parts.  Since  these  eigenvalues  may  have  moderately  large  real 

2 

parts  [they  can  be  as  large  as  U /2v  by  (12.4)],  there  may 
be  rapid  growth  of  errors  and  numerical  solution  of  the 
Chebyshev-spectral  equations  may  appear  unstable  and  divergent. 

For  N ^crit  ' fc^ere  are  no  eigenvalues  of  with  positive 

real  parts  so  the  spectral  equations  are  stable. 

The  origin  of  this  temporal  instability  is  the  outflow 
boundary  layer  at  x = ±1  ; when  U > 0,  the  solution  to 
(12.1-3)  develops  a region  of  rapid  change  of  width  roughly 

1/2 

v/U  near  x = +1  as  t increases.  Since  roughly  3(U/v) 

Chebyshev  polynomials  are  required  to  resolve  a boundary  layer 

1/2 

of  width  v/U  [see  (3.50)],  we  expect  that  Ncrit  t 3(U/v)  ' 

2 

so  VN  . ,/U  + 9 . In  fact,  as  shown  in  Table  12.1,  the 
cnt 

2 

criterion  is  actually  vNcr^t/U  ^ 4*  [Since  the  Chebyshev  norm 

1/4 

of  exp(-Ut3/9x)  is  roughly  N (see  Sec.  8),  we  expect 

that  the  proper  scaling  of  Ncrit  is  better  represented  as 
7 /4 

vNcrj^/U  4 1.3.  As  shown  in  Table  12.1,  this  modified  scaling  is 
more  nearly  satisfied  for  the  range  of  v considered.] 
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TABLE  12.1 


v/U 

Ncrit 

crit/0 

vN7/4 

crit 

1.0  x 10-2 

15 

2.25 

1.14 

2.5  x 10-3 

35 

3.06 

1.26 

1.0  x 10“3 

61 

3.72 

1.33 

6.0  x l(f4 

81 

3.94 

1.31 

1 

O 

H 

X 

o 

• 

101 

4.08 

1.29 

Table  12.1  Critical  values  Ncrit  of  the  number  of  Chebyshev 
polynomials  necessary  that  the  tau  approximation  to  the  operator 
-U3u/3x  + v3  u/3x  with  u(±l)  = 0 have  no  eigenvalue  with 

positive  real  parts.  Also  listed  are  the  inverse  'grid  Reynolds 

2 7/4 

number'  vN^rit/U  and  the  Parameter  vNcrit^U 
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If  Chebyshev- spectral  approximations  to  (12.1-3)  are 

solved  using  fractional  time-step  methods,  the  temporal 

instability  for  N < Ncrit  appears  in  a unique  way.  Define 

the  operator  as  an  N-mode  Chebyshev  approximation  to  the 

operator  -U3u/3x  with  the  boundary  condition  u(-l)  = 0 

and  the  operator  BN  as  an  N-mode  Chebyshev  approximation 

2 2 

to  the  operator  v3  u/3x  with  u(±l)  = 0.  Then  the 
evolution  operator  of  (12.1-2)  is  exp ( (A^B^  t]  so  a 
fractional  step  method  involves  the  splitting 
3uN/3t  = 3xuN/3t  + 32uN/3t  where 

3lV3t  = Vn  ' 32UN/3t  = BNUN  * 

For  any  values  of  v and  U > 0,  the  fractional  step 
S^Ujj/dt  is  algebraically  stable  since  ||exp  A^H  = 0(N1//4) 
(see  Sec.  8) , while  the  fractional  step  32uN/3t  is  stable 
since  ||exp  B^t ||  £ 1 (see  Sec.  7).  Nevertheless, 

||  exp  I (Aj^+Bjj)  t]  ||  can  grow  rapidly  with  t.  The  reason  is 
that  ^ and  Bn  do  not  commute  so  it  is  not  true  that 
HexpKAjj+B^t]  ||  £ ||exp  Ajjtll  ||exPBNt|| 

The  Lie  formula  (5.8)  does  ensure  that 

||  exp  [ (Ajg-'-Bj^)  t]||  < lim  ||  exp  (J^t/n)  ||  n ||  exp  (BNt/n  ||  n. 

n-*-°° 
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However,  as  n ■*  », 


HexpJAjjt/n)  ||  - 1 'v  Cn  t/n 


with  C > 0 (see  Sec.  8)  so 


exp  (Ajjt/n)  || n 'v  exp(CN2t)  >>  1 


(n  -*■  «)  ; 


Therefore  the  Lie  formula  gives  only  the  very  weak  upper  bound 


||  exp  [ (Ajj+B^  t]  ||  < exp(CN2t) 


In  summary,  Chebyshev-spectral  approximations  to  (12.1-3) 

give  fractional  step  methods  such  that  each  fractional  step 

is  algebraically  stable  while  the  total  step  is  unstable 

unless  N > N ... 

cr  it 

If  the  boundary  conditions  (12.2)  are  replaced  by 


u(-l,t)  = 


(+l,t)  = 0 


(12.4) 


when  U > 0,  the  criterion  for  temporal  stability  is  relaxed 

2 

significantly.  As  shown  in  Table  12.2,  the  value  of  vNcrit/U 
is  decreased  to  roughly  1.6.  However,  the  growing  modes  that 
appear  when  N < Ncr^t  are  much  tamer  than  those  that  appear 
when  the  boundary  condition  u(+l,t)  =0  is  applied,  so 
accurate  time  integrations  can  still  be  obtained  when 
vN2/U  % 0.01  (see  Haidvogel  1977). 
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TABLE  12.2 


v/U 

N 

crit 

VN7/4  . 
crit 

2.5  x io"3 

21 

0.52 

1.0  x 10-3 

37 

0.56 

6.0  x 10-4 

49 

0.54 

1 

o 

rH 

X 

O 

• 

61 

0.53 

2.0  x 10“4 

89 

0.52 

Table  12.2.  Critical  values  N . . of  the  number  of 

crxt 

Chebyshev  polynomials  necessary  that  the  tau  approxima- 

2 2 

tion  to  the  operator  -U3u/3x  + v3  u/3x  with 

u(-l)  = 0,  3u(+l)/3x  = 0 and  U > 0 have  no 

eigenvalues  with  positive  real  parts.  The  parameter 
7/4 

vNc^.it/U  is  also  listed. 


13.  Models  of  Incompressible  Fluid  Dynamics 


The  Stokes  equations  for  low  Reynolds  number,  two- 
dimensional  incompressible  flow  are 


!£  - - ?P  ♦ w2v. 

$.V  = 0 , 


(13.1) 


where  v is  the  velocity  field,  p is  the  pressure,  and  v is 
the  kinematic  viscosity.  With  the  boundary  conditions  that 
v = 0 on  rigid  stationary  boundaries,  the  problem  (13.1)  is 
well  posed  for  any  v > 0.  An  equivalent  formulation  is  given 
by  the  vorticity-streamfunction  equations 


£1  = 
3t 


W2S  , 


(13.2) 


C = V yi  , 


obtained  by  taking  the  curl  of  the  Stokes  equations  (13.1).  Here 
ij;  is  the  streamfunction  defined  by  v = (-9i|i/9y , 9ip/9x)  and  c 
is  the  vorticity. 

A one-dimensional  model  of  (13.2)  is 

= v ^4  (-1  < x < 1,  t > 0),  (13.3) 

9x^  “ “ 

2 

C = ^-4  (-1  < x < 1).  (13.4) 

9x^  ~ “ 

On  stationary  rigid  walls,  the  boundary  conditions  for  (13.3-4)  are 


(x , t)  = i|>x(x,t)  =0  (x  = ±1). 


(13.5) 


There  is  one  subtlety  in  the  application  of  spectral  methods 
to  (13.3-5)  that  does  not  appear  directly  when  the  primitive 
equations  (13.1)  are  used.  It  is  necessary  to  use  some  care 
to  avoid  unconditional  numerical  instability  with  the  Chebyshev- 
tau  method. 
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r 


f! 

* 

i* 


The  most  obvious  way  to  use  the  tau  method  to  solve 
(13.3-5)  is  to  substitute  (13.4)  into  (13.3)  and  solve 

*xxt  “ v*xxxx  (-1  < x < 1,  t > 0) 
by  expanding  i^(x,t)  in  the  Chebyshev  series 

N 


(13.6) 


ijj(x,t)  = I a (t)T  (x)  . 

A " " 


(13.7) 


n=0 


Denoting  by  a^  the  Chebyshev  expansion  coefficients  of  3^^/9x^ 


(see  A. 20),  the  tau  equations  for  (13.5-6)  are 

(2) 

(4)  (0  < n < N-4,  t > 0) , (13.8) 


da 

n 

St 


n 


N N - 

l <±l>na_  = l (±Dnn  a = 0. 

n=0  n=0 


(13.9) 


Unfortunately,  this  method  for  solution  of  (13.3-5)  is 
unconditionally  unstable  as  N •+■  ®.  In  Table  13.1,  we  list 
the  largest  positive  eigenvalue  A of  (13.8-9);  there  is 
a solution  of  (13.8-9)  that  grows  like  a (t)  = a^ (0 ) exp (A__wt) . 

II  n ITlaX 

4 

Since  Amax  grows  like  N as  N + ®,  errors  also  grow  rapidly 
as  H + « for  fixed  t.  This  method  is  unusable  for  time- 
dependent  calculations. 

In  Table  13.1,  we  also  list  the  values  of  An  for  n = 1,5, 

where  the  eigenvalues  of  (13.8-9)  are  ordered  according  to 

| A.  | < |X2I  £ ...  . The  exact  eigenvalues  of  (13.3-5)  are 

found  by  seeking  solutions  of  these  equations  of  the  form 

i|;(x,t)  = ^(x)exp(At),  £(x,t)  = x,  (x)exp(At)  . It  may  be  easily 

verified  that  the  exact  eigenvalues  of  (13.3-5)  are  given  by 
2 

A = -y  with  y = nir  or  y any  nonzero  solution  of  the  transcendental 
equation  tan  y = y.  The  exact  values  of  A^  and  A^  are  also  listed 
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Table  13.1 


N 

X 

A 

X 

1 

5 

max 

10 

-9.8696598 

-189.63800 

4,272. 

15 

-9.8696044 

- 89.54550 

29,439. 

20 

-9.8696044 

- 88.86244 

111,226. 

25 

-9.8696044 

- 88.86244 

294,697. 

: 30 

-9.8696044 

- 88.86244 

652,722. 

35 

-9.8696044 

- 88.86244 

1,255,298. 

40 

-9.8696044 

- 88.86244 

2,215,880. 

Exact 

-9.8696044 

- 88.86244 

Table  13. 

1.  Eigenvalues  of  the  tau  approximation  (13.8-9) 

to  (13.6- 

7).  The  N-4 

eigenvalues  are  ordered 

so  that 

|XjJ  | X 2 | £ ...  | An_4  | .All  the  eigenvalues  are  real. 

The  largest  positive  eigenvalue  Amax  = *n-4* 
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in  Table  13.1.  Evidently,  even  thouqh  (13.8-9)  is  unstable  as 
N -*■  it  does  a good  job  of  reproducing  the  low-n  modes; 
approximately  I Chebyshev  polynomials  are  required  to 

resolve  the  mode  with  eigenvalue  * . Thus,  this  version 
of  the  tau  method  may  be  useful  for  eigenvalue  calculations 
even  though  it  is  unconditionally  unstable  for  the  initial- 
value  problem  (13,3-5)  (as  evidenced  by  the  spurious  unstable 
modes  with  eigenvalues  as  large  as  ^max) . 

The  tau  method  behaves  similarly  when  applied  to  more 
difficult  problems,  like  the  Orr-Sommerf eld  equation  for 
linear  stability  analysis  of  incompressible  plane-parallel 
shear  flows.  Low  modes  are  given  accurately  by  the  analog 
of  (13.8-9)  (see  Orszag  1971  ),  but  there  appear  spurious 
unstable  modes  with  large  growth  rates.  Similar  spurious 
unstable  modes  appear  in  finite-difference  solution  of  the 
Orr-Sommerfeld  equation  (see  Gary  & Helgason  1970). 

There  is  a simple  method  to  avoid  the  spurious  unstable 
modes  encountered  by  (13.8-9).  The  technique  to  be  described 
below  also  eliminates  the  spurious  unstable  modes  encountered 
in  solution  of  the  Orr-Sommerfeld  equation.  The  idea  is 
simply  not  to  combine  (13.3-4)  into  (13.6).  Rather,  we 
expand  c(x,t)  as  in 

N 

t(x,t)  = l b ( t ) T (x ) (13.10) 

n=0  n n 

and  solve 


dbn 

~ai 

■ « <2) 

(0 

_<  n _<  N-2 ) , 

(13. 

ID 

bn 

- ^2> 

(0 

n _<  N-2)  , 

(13. 

12) 

. Here  we  have  dropped  two  equations 
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in  addition  to  (13.9) 


( 


l ' 


1 1 


II 


e 


(! 


( 


from  the  Chebyshev  modal  equations  that  result  from  (13.3-4). 

The  logic  of  this  modification  of  the  tau  method  is  as  follows. 
Application  of  (13.8)  for  0 <_  n <_  N-4  is  equivalent  to 
application  of  (13.12)  for  0 _<  n <_  N together  with  (13.11)  for 
0 < n <_  N-4.  On  physical  grounds,  we  may  expect  that  this 
procedure  will  lead  to  instability  because  the  boundary  conditions 
iji  * 0 at  x = +1  should  be  imposed  on  (13.4)  not  (13.3),  while 
the  boundary  conditions  ^ = 0 at  x = il  should  be  imposed  on 
(13.3)  only  when  v > 0.  On  the  other  hand,  when  the  system 
is  truncated  as  in  (13.11-12),  each  of  the  dynamical  equations 
can  play  their  proper  role  in  adjusting  the  boundary  conditions: 
the  boundary  conditions  = 0 are  imposed  on  (13.12)  while 
the  boundary  conditions  i^x  = 0 are  imposed  on  (13.11). 

We  shall  now  prove  that  (13.11-12)  is  stable  for  the 
special  case  in  which  N is  even  with  a2n+l  = b2n+l  = 0 for 
all  n,  t 21  0.  In  this  case,  i|i(x,t)  and  t(x,t)  are  even  functions 
of  x.  To  begin  , we  observe  that  (13.11)  is  equivalent  to 

(-1  < X < 1.  t>0)  , 

while  (13.12)  is  equivalent  to 

2 

C(x,t)  « — £ + b T (x)  . 

3x 

Therefore, 


T ♦ 


v y ♦ Vi1 


atax 

Since  is  an  even  function  of  x,  it  follows  by  integration  with 
respect  to  x that 


a 2u> 
axat 


Vn 


(13.13) 


( 
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Also,  since  t/>(x,t)  is  a polynomial  of  degree  N that  satisfies 
^x(±l,t)  = 0,  integration  by  parts  gives 


/ ^xT^(l-x2)_,5dx  = 


' / l*xx  + x*x/(l"X2)]TN{l-x2)'%dx  = 0 


since  »|>xx  and  x^x/ (l-x“ ) are  polynomials  of  degree  N-2  so  they  must 
be  orthogonal  to  TN(x).  Therefore,  taking  the  Chebyshev  inner 
product  of  (13.13)  and  ^x(x,t),  we  obtain 


7t  / <(l-x2)"Hox  = 2 v / ^x*xxx(l-x‘)  ’dx  < 0,(13.14) 


2^-H, 


where  the  last  inequality  is  established  using  the  inequality 

derived  in  Example  7.1(v): 

1 2 — k 

/ uuxx(l-x  ) Sdx  <_  0 

if  u(x)  i a polynomial  of  degree  N satisfying  u(±l)  =0.  The 
energy  bound  (13.14)  proves  stability  of  the  tau  approximation 
(13.11-12) . 


Finally,  let  us  discuss  methods  for  the  solution  of 
the  primitive  equations  (13.1)  using  Chebyshev  tau  approximations. 

A one-dimensional  model  that  embodies  the  essential  features 
of  (13.1)  is  obtained  by  solving  (13.1)  within  the  slab  -1  £ x <_  1, 

“°°  £ y £.  00 , with  an  assumed  solution  of  the  form 

v = (u(x,t)eiky,  v(x,t)eiky),  p = p(x,t)eiky 
for  some  real  wavenumber  k.  Let  the  Chebyshev  expansion 
coefficients  of  u(x,t),  v(x,t),  p(x,t)  be  denoted  as  un(t), 
vn(t),  Pn(t)  (0  < n <_  N)  , respectively.  Then  an  unconditionally 
stable,  implicit  fractional  step  method  for  the  solution  of  (13.1) 
with  a forcing  term  (f (x,t)eiky ,g (x, t)eiky)  added  to  the  right  side  is 
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"n  " un(t)  + Atf"Pn1>  + fn(t>1  (0  - 
% " vn(t)  + AtI”ikPn  + 9n(t)^  (0  - 

u„(1)  + ikv  =0  (0  < 

n n — 


N N 

l un  - l un  “ 0 i°l 

n=0  n=0 

u (t+At)  - vAtu!2) (t+At)  « u„  (0  < 
n n n — 

(t+At)  - vAtv^  (t+At)  = vn  (0  £ 


N N 

l ( l)nu  (t+At)  = l ( l)nv  (t+At) 
n=0  n=0 


n 

n 

n 

n 

n 

n 


1 N-2 ) , 

< N)  , 

< N)  , 

< N)  , 

< N-2)  , 

< N-2)  , 

0. 


(13.15) 

(13.16) 

(13.17) 

(13.18) 

(13.19) 

(13.20) 

(13.21) 


12) 

Here  we  use  the  notation  that,  for  example,  represents 

the  Chebyshev  coefficients  of  uxx(x,t).  The  scheme  (13.15-21) 
is  based  on  backwards  Euler  time  differencing;  it  is  straight- 
forward to  generalize  (13.15-21)  to  other  more  accurate  time 
differencing  methods. 

The  fractional  step  (13.15-18)  involves  computation 
of  the  pressure  field  by  imposition  of  the  incompressibility 
condition  (13.17).  Only  the  boundary  conditions  u(±l,t)  = 0 
are  applied  because  this  part  of  the  time  step  is  effectively 
inviscid  so  only  the  normal  flow  can  be  specified  at  the 
boundary.  Thus,  we  drop  (13.15)  for  n = N-1,N  in  favor  of 
the  two  boundary  conditions  (13.18).  The  fractional  step 
(13.19-21)  involves  the  viscous  term  in  (13.1)  so  boundary 
conditions  are  applied  on  both  the  normal  velocity  component 
u and  the  tangential  velocity  component  v.  Accordingly,  the 
tau  method  involves  droppinq  (13.19-20)  for  n » N-1,N  in  favor 
cf  these  boundary  conditions. 
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The  system  (13.15-21)  is  solved  as  follows.  Multiplying 
(13.15)  by  ik  and  subtracting  the  result  from  the  Chebyshev 
x-derivative  of  (13.16)  gives 

vn(1)  - ikun  = (t)  “ ikun(t)  + At[gn^(t)  - ikfn<t)] 

(0  <_  n < N-2)  . 

Substituting  vr  = iun  ^ /k  from  (13.17)  gives 

u„  ( 2 } -k2u  = u[2)  (t)-k2u  (t)  + At  [-ikg,!1)  (t)  - k2f  (t)  ] 
n n n _ n n n 

(0  <_  n < N-2)  . (13.22) 

Eq.  (13.22)  with  the  boundary  conditions  (13.18)  is  of  the 
same  form  as  (13.19-20)  with  boundary  conditions  (13.21). 

These  equations  are  best  solved  by  the  algorithm  discussed 
at  the  end  of  Sec.  10. 

The  stability  analysis  of  (13.15-21)  is  as  follows. 

The  evolution  of  a perturbation  is  governed  by  (13.15-21) 

with  f = g =0  for  all  n.  Therefore,  the  solution  of  (13.22) 
n n 

is  u = u (t)  for  all  n.  Also,  v = v (t)  for  all  n.  Finally, 
n n n n 

the  implicit  scheme  (13.19-21)  is  an  unconditionally  stable 
scheme  for  solution  of  the  heat  equation.  This  proves  that 
(13.15-21)  is  unconditionally  stable. 

The  methods  discussed  in  this  section  extend  to  give 
stable  methods  for  solution  of  the  nonlinear  Navier-Stokes  equations. 
For  example,  if  the  forcing  term  (f,g)  in  (13.15-16)  is  chosen 
to  be  the  nonlinear  terms  of  the  Navier-Stokes  equations,  our 
analysis  shows  that  stability  of  (13.15-21)  is  determined  by  stability 
restrictions  on  the  nonlinear  terms  alone. 


14 . Miscellaneous  Applications  of  Spectral  Methods 


In  this  Section,  we  survey  some  special  topics  regarding 
spectral  methods.  Some  of  these  topics  are  still  under  active 
investigation,  so  the  results  reported  here  are  very  incomplete. 

Complicated  Geometries 

There  are  two  ways  that  spectral  methods  can  be  used 
to  solve  problems  in  complicated  geometries  without  introducing 
basis  functions  that  are  special  to  the  geometry  and,  therefore, 
unwieldy  and  inefficient  to  use.  The  two  methods  are  mapping 
and  patching. 

Mapping  involves  transforming  the  complicated  domain 
into  a simpler  one  by  means  of  a coordinate  transformation. 
Spectral  methods  are  then  applied  in  the  simple  geometry 
using  the  techniques  discussed  in  earlier  sections.  For 
example,  if  we  wish  to  solve  the  heat  equation 

u (x,y , t)  = V2u(x,y,t)  (14.1) 

in  the  two-dimensional  domain 

-i  < x < i,  -f(x)  < y < f (x) 
for  some  given  function  f(x)  with  the  boundary  conditions 
that  u = 0 on  the  boundary  of  the  domain,  we  would  proceed 
as  follows.  First,  we  make  the  coordinate  transformation 

2 * y/f (x)  D (14.2) 

and  rewrite  (14.1)  as 

2 

~ u(x,z,t)  = (-^  “ f 2 ^-)2u(x,z  ,t)  + f 2— u(x,z,t) 

9 z 

(-1  < x < 1,  -1  <_  z <_  1)  . (14.3) 
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Then,  we  expand  u(x,z,t)  in  a double  Chebyshev  series 
and  integrate  (14.3).  For  this  purpose,  a hybrid  numerical 
scheme  is  suggested  in  which  time  differencing  is  stabilized 
by  a semi-implicit  method  (see  Sec.  10)  in  which  a simple 
diffusion  operator  is  added  and  subtracted  from  (14.3).  The 
simple  diffusion  operator  is  then  evaluated  using  a tau 
method  (because  the  tau  method  is  simplest  when  no  complicated 
nonlinearities  or  nonconstant  coefficient  terms  are  involved) ; 
the  remaining  nonconstant  coefficient  term  in  (14.3)  is  then 
evaluated  using  fast  Fourier  transforms  and  the  collocation 
method.  The  resul'  is  an  efficient  and  accurate  method  for 
solution  of  (14.1). 

Techniques  like  those  just  described  have  been  applied 
at  a variety  of  problems  with  much  success.  If  a convenient 
coordinate  transformation  is  available,  the  mapping  technique 
combined  with  appropriate  spectral  methods  may  be  expected 
to  be  very  useful. 

The  idea  of  patching  is  that  if  the  geometry  is  the 
union  of  several  simpler  geometries  (like  an  L-shaped  region) 
then  spectral  approximations  can  be  formulated  in  each  of 
the  simpler  domains  and  then  patched  across  the  boundaries  by 
requiring  that  the  solution  (and  an  appropriate  number  of 
derivatives)  be  smooth.  When  this  technique  is  applied 
together  with  the  mapping  technique  discussed  above,  it  is 
possible  to  devise  spectral  shock-fitting  methods  for  the 
solution  of  compressible  flow  problems.  These  methods  require 
much  further  investigation  to  judge  their  usefulness  in  practical 
problems . 


Poisson '■  Equation  in  Two  and  Higher  Dimensions 


V2u 


The  Chebyshev  tau  equations  for  Poisson's  equation 
f in  the  square  -1<x<1  , -l<y^l  are 

u(2'0)  + u,J°'2)  = f ( 0 <n <N- 2 , 0 <m <M- 2 ) , 

run  nm  nm  — — — — 

whi le  the  Dirichlet  boundary  conditions  u = 0 are 


N 


l unnu  = 0 
L nm 


n=0 


N 


Z = 0 


m=0 


nm 


(0<m<M) 


(0<n<N) 


(14.4) 


(14.5) 


(14.6) 


Here  we  expand  u(x,y)  and  f(x,y)  in  the  double  Chebyshev  series 

N M u. 


ru  x,yn  = l l {f  } T (x ) T (y)  (14.7) 

'f(x*y)J  n£0  m£0  irnmJ  n v m^7 

and  we  denote  the  Chebyshev  expansion  coefficients  of  3^+<^u/3x^9y^ 

by  u<P'q).  The  2N+2M+4  boundary  conditions  are  not  all  linearly 
nm 

independent;  there  exist  four  linear  relations  among  them,  namely 

(14.8) 


N M „ 

l l (±1) n (±l)mu  = 0. 

n=0  m=0  n 


Thus,  (14.4-6)  gives  (N+l)  (M+l)  equations  for  the  (N+l) (M+l) 

unknowns  u _ (0<n<N,  0<m<M) . 

nm  — — — — 

Using  (10.7)  [or  (A.20)],  the  system  (14.4-6)  can  be  reduced 

to  a block  tridiagonal  matrix  equation  modified  by  extra  full 

rows  corresponding  to  the  boundary  conditions  (14.5-6).  These 

equations  can  be  solved  by  standard  block  tridiagonal  algorithms 
3 3 

in  order  N M or  order  NM  operations.  If  Poisson's  equation  must 
be  solved  several  times  with  the  same  values  of  N and  M but  different 
functions  f(x,y),  it  is  more  efficient  to  apply  alternative  methods. 
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2 

A method  to  solve  (14.4-6)  in  order  N M operations  (with 


a preprocessing  stage  that  requires  order  N3  operations)  is 
as  follows.  First,  we  find  the  N-2  eigenvalues  X^  and  eigei 
vectors  e (p  = 0,...,N-2)  of  the  equations 


enp’  - Vnp  N-2) 

" „(11,nenp  * °- 
n=0 

The  eigenvalues  Xp  are  all  negative  as  proved  in  Example  7.3  (ii)  . 
Then  we  form  the  (N+l)x(N+l)  matrix  E whose  elements  are 


Enp  = enp  (0  < n < N,  0 < p < N-2) 

En,N-l  - Sn,0  <0  i n i "I 
En.N  = 5n,l 

and  compute  the  inverse  matrix  D = E-i.  Since  the  boundary 

conditions  (14.5)  are  satisfied  by  u , it  follows  that 

nm 

N-2 

u = ; e v 

nm  J;0  np  pm 

for  suitable  v for  all  n,m.  Therefore,  setting 
pm 

N 

g„  = l (D)  f _ (0<p< N-2,  0 <m <M- 2 ) , (14.10) 
pm  n pn  nm  —r—  — — 

n=u 


(14.9) 


it  follows  that  (14.4-6)  become 


X v + v^°'2)  = g (0 <p<N-2 , 0 <m <M- 2 ) (14.11) 

p pm  pm  pm  — — — — 


M 


t (±l)n'vpin  = 0 (0<p<N-2)  . 


m-0 


(14.12) 


Eqs.  (14.11-12)  may  be  solved  efficiently  (in  order  NM  operations) 

for  vpjn  using  the  algorithm  discussed  at  the  end  of  Sec.  10. 

Once  v is  found,  u may  be  reconstructed  from  (34.9).  The 
pm  ’ nm 

2 

total  operation  count  is  order  N M [from  the  two  matrix  multiplies 
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The  solution  of  Poisson's  equation  by  the  Chebyshev 
series  method  outlined  above  is  very  competitive  with  finite- 
difference  solution  using  fast  Poisson  solvers.  Zang  & 

Haidvogel  (1977)  present  a number  of  comparisons  of  the  Chebyshev 
methods  and  fast  Poisson  solvers. 

There  are  two  further  complications  that  may  arise  in 
elliptic  boundary-value  problems.  First,  the  elliptic  equation 
may  have  nonconstant  coefficients  or  may  even  be  nonlinear. 

Here  we  recommend  that  spectral  equations  be  solved  using 
relaxation  methods  of  the  kind  advocated  by  Concus  & Golub  (1973), 
in  which  the  heart  of  the  algorithm  is  the  fast,  efficient 
solution  of  Poisson-like  equations.  Second,  the  geometry 
may  be  more  complicated  than  a box.  In  this  case,  we  recommend 
the  implementation  of  capacitance  matrix  techniques  (or 
equivalent  Green's  function  techniques)  in  which  the  problem 
to  be  solved  is  imbedded  in  a simpler  geometry,  like  a box 
(see  Buzbee  et  al  1971) . Again,  the  heart  of  the  algorithm 
is  the  fast  solution  of  Poisson's  equation  using  (14.9-12). 

Coordinate  Singularities 

When  spectral  methods  are  applied  to  problems  in 
cylindrical  or  spherical  geometries,  their  formualtion  may 
require  special  care  at  the  coordinate  singularities.  These 
'pole  problems'  have  been  extensively  investigated  (Orszag  1974, 
Tang  1977).  As  a simple  example  of  these  effects,  let  us 
consider  the  computation  of  the  eigenvalues  of  Bessel's  equation 
using  the  Chebyshev  tau  method  (Metcalfe  1974).  The  problem  is 


r ■ 


to  find  the  eigenvalues  and  eigenfunctions  y(x)  of 


y"  + jy'  - jjy  y ■ - *y 


(14.13) 


subject  to  the  conditions  that  y ( 1 ) = 0 and  that  y(x)  be  finite 

for  0 ^ x ^ 1.  The  exact  eigenvalues  are  related  to  the  zeros  of  the 

2 

Bessel  function  Jn:  = jnp  where  Jn(3np)=0,  p=l,2,...  . 

When  n is  even,  the  eigenfunctions  of  (14.13)  are  even 
functions  of  x;  when  n is  odd,  the  eigenfunctions  are  odd. 

This  fact  suggests  that  we  represent  the  solution  to  (14.13) 
in  terms  of  series  of  even  Chebyshev  polynomials  when  n is  even 
and  odd  polynomials  when  n is  odd.  Thus,  for  n odd  we  write 

y(x)  = I YmT2m.1(x)  . (14.14) 

m=l 

In  Table  14.1,  we  list  numerical  values  for  the  smallest  eigenvalue 

of  (14.13)  with  n = 7 using  the  series  (14.14),  the  boundary 

condition  y(l)  = 0,  and  the  Chebyshev  tau  method.  The  convergence 

of  this  method,  while  very  impressive  as  M increases,  is  slowed 

by  the  coordinate  singularity  of  (14.13)  at  x = 0 . In  general, 

series  of  the  form  (14.14)  behave  like  x as  x -*  0.  In  this 

2 

case  the  terms  y'/x  and  y/x  are  singular  at  x =0.  The  true 

7 

eigenfunctions  J7(jn7x)  behave  like  x as  x + 0,  as  may  easily 
be  shown  using  Frobenius'  method,  so  none  of  the  terms  of  (14.13) 
are  in  fact  singular  for  the  exact  eigenfunctions. 

It  is  possible  to  improve  the  convergence  of  (14.14)  by 
imposing  additional  'pole  conditions',  like  y' (0)  = 0.  When 
y'(0)  = 0 in  the  series  (14.14),  the  terms  of  (14.13)  are 
individually  nonsingular.  In  Table  14.1,  we  also  list  numerical 
values  of  the  smallest  eigenvalue  of  (14.13)  with  n = 7 and 
the  two  boundary  conditions  y(l)  = 0,  y'(0)  = 0 applied.  There 
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Table  14.1 


M 

with  y (1)  =0 

X^  with  y(l)=y 

10 

124.001290649 

14 

169.111983340 

122.895944051 

18 

126.557832251 

122.907620295 

22 

122.991799598 

122.907600279 

26 

122.908250800 

122.907600204 

Exact 

122.907600204 

122.907600204 

Table  14.1.  Smallest  eigenvalue  of  (14.13)  with  n = 7 
obtained  using  (14.14)  and  the  Chebyshev  tau  method. 

M is  the  number  of  Chebyshev  polynomials.  The  extra 
boundary  condition  y' (0)  = 0 is  a pole  constraint  at 
the  singular  point  x = 0 of  (14.13). 
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is  clearly  a dramatic  improvement  in  the  rate  of  convergence. 

It  is  also  possible  to  make  the  problem  less  sensitive  to 

pole  properties  near  the  origin  by  first  multiplying  (14.13) 

2 

by  x to  eliminate  explicitly  singular  terms  and  then  applying 
the  tau  method.  The  results  of  the  latter  trick  are  essentially 
the  same  as  applying  the  pole  condition  y'  (0)  = 0 directly  to 
(14.13) . 


If  pole  conditions  are  not  properly  applied,  it  is  possible 
to  degrade  significantly  the  accuracy  of  spectral  computations. 
It  is  even  possible  to  induce  strong  instabilities  that  are 
absent  when  proper  pole  conditions  are  applied.  These  matters 
are  discussed  in  detail  by  Orszag  (1974)  and  Tang  (1977). 
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15 . Survey  of  Spectral  Methods  and  Applications 


In  this  Section,  we  give  a brief  survey  of  spectral 
methods  and  some  of  their  recent  applications.  There  are 
five  important  features  of  spectral  methods  that  should  be 
considered  in  their  formulation  and  application.  They  are: 

(i)  Rate  of  convergence  - If  the  solution  to  a problem 
is  infinitely  differentiable,  then  a properly  designed 
spectral  method  has  the  property  that  errors  go  to  zero 
faster  than  any  finite  power  of  the  number  of  retained  modes. 

In  contrast,  finite-difference  and  finite-element  methods 
yield  finite-order  rates  of  convergence.  The  important 
consequence  is  that  spectral  methods  can  achieve  high  accuracy 
with  little  more  resolution  than  is  required  to  achieve  moderate 
accuracy. 

(ii)  Efficiency  - The  development  of  fast  transform 
methods  permits  spectral  methods  to  be  implemented  with 
comparable  efficiency  to  that  of  finite  difference  methods 
with  the  same  number  of  independent  degrees  of  freedom. 

However,  since  spectral  methods  typically  require  a factor 
of  2-5  fewer  degrees  of  freedom  in  each  space  direction  to 
achieve  moderate  accuracy  (say,  5%  error),  the  spectral 
computations  can  be  considerably  more  effective.  As  the 
required  accuracy  increases,  the  attractiveness  of  spectral 
methods  increases. 

(iii)  Boundary  conditions  - As  shown  in  earlier  Sections 
of  this  monograph,  the  mathematical  features  of  spectral 
methods  follow  very  closely  those  of  the  partial  differential 
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equation  being  solved.  Thus,  the  boundary  conditions  imposed 
on  spectral  approximations  are  normally  the  same  as  those 
imposed  on  the  differential  equation.  In  contrast,*  finite- 
difference  methods  of  higher  order  than  the  differential  equation 
require  additional  'boundary  conditions.'  Many  of  the 
complications  of  finite-order  finite-difference  methods  disappear 
with  the  infinite-order-accurate  spectral  methods. 

Another  aspect  of  the  treatment  of  boundary  conditions 
by  spectral  methods  is  their  high  resolution  of  boundary 
layers.  If  the  solution  to  a problem  has  a boundary  layer 

L 

of  thickness  e , then  only  about  1/r^  polynomials  [see  (3.50)1  need 
be  retained  to  achieve  high  accuracy.  In  contrast,  finite- 
difference  methods  using  equally  spaced  grid  points  would  require 
about  1/e  grid  points  to  resolve  such  a boundary  layer  solution. 
Moreover,  if  a coordinate  transformation  is  employed  to  improve 
the  resolution  of  a boundary  or  internal  layer  of  thickness  r , 
the  errors  of  spectral  methods  are  decreased  faster  than  any 
finite  power  of  e as  e -*•  0. 

(iv)  Discontinuities  - Surprisinqly , spectral  methods 
do  a better  job  of  localizing  errors  than  difference  schemes 
and  hence  require  considerably  less  local  dissipation  to  smooth 
discontinuities . 

(v)  Bootstrap  estimation  of  accuracy  - It  is  often 
possible  to  estimate  the  accuracy  of  spectral  computations 

by  examination  of  the  shape  of  the  spectrum.  Thus,  in  computations 
of  three-dimensional  incompressible  flows  at  high  Reynolds  numbers, 
the  mean-3quare  vorticity  spectrum  must  not  increase  abruptly  at 
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large  wavenumbers  (small  scales);  if  the  vorticity  spectrum 
decreases  smoothly  to  0 as  wavenumber  increases,  it  is  safe 
to  infer  that  the  calculation  is  accurate.  On  the  other  hand, 
similar  criteria  for  finite-difference  methods  can  be  very 
misleading. 

Let  us  now  survey  some  applications  of  spectral  methods 
to  incompressible  fluid  dynamics.  We  shall  classify  the  method 
according  to  the  boundary  conditions  and  geometry. 

(i)  Periodic  boundary  conditions  in  Cartesian  coordinates  - 
Here  Fourier  series  are  appropriate.  Spectral  methods  have 
been  regularly  used  in  three  dimensions  with  32  x 32  x 32 
modes  and  in  two  dimensions  with  128  x 128  modes  to  simulate 
homogeneous  turbulence.  Most  operational  codes  now  use 
pseudospectral  (collocation)  methods  because  aliasing  errors 
are  usually  small.  The  key  fast  transform  methods  are  described 
in  detail  by  Orszag  (1971c) . 

More  recently,  more  ambitious  spectral  codes  have  been 
developed.  The  KILOBOX  code  employs  1024  x 1024  Fourier  modes 
in  two  dimensions  while  the  CENTICUBE  code  uses  up  to 
128  x 128  x 128  modes  in  three  dimensions.  These  high  resolution 
codes  are  now  being  used  to  study  fundamental  questions 
regarding  high  Reynolds  number  turbulence,  including  the  structure 
of  inertial  ranges. 

(ii)  Rigid  boundary  conditions  in  Cartesian  coordinates  - 

Here  Chebyshev  polynomials  should  be  employed.  Typical 

applications  to  date  include  numerical  studies  of  turbulent 
shear  flows  and  boundary  layer  transition.  Pseudospectral 
methods  are  used,  with  Chebyshev  polynomials  particularly 
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► 

convenient  because  fast  Fourier  transform  methods  can  be 
applied. 

(iii)  Rigid  boundary  conditions  in  cylindrical  geometry  - 
Here  Chebyshev  polynomials  should  be  used  in  radius,  Fourier 
series  in  angle,  and  either  Fourier  or  Chebyshev  series  in 
the  axial  direction  (depending  on  boundary  conditions).  Some 
technical  aspects  of  the  implementation  of  Chebyshev  series 
in  radius,  including  pole  conditions,  is  discussed  by  Orszag 
(1974).  Applications  to  date  include  studies  of  transition 
in  circular  Couette  flow  and  pipe  Poiseuille  flow.  In  particular, 
it  should  be  emphasized  that  Chebyshev  polynomial  expansions 
are  much  better  suited  for  serious  numerical  work  than  the 
apparently  more  natural  choice  of  Bessel  function  expansions 
in  radius.  There  are  two  reasons:  Chebyshev  series  converge 
faster  to  general  functions  regardless  of  their  boundary 
conditions  and  Chebyshev-spectral  methods  can  be  implemented 
efficiently  by  fast  transform  methods. 

(iv)  Problems  in  spherical  geometry  - Here  surface 
harmonic  expansions,  generalized  Fourier  series,  and  'associated' 
Chebyshev  expansions  all  have  attractive  features.  A 
detailed  discussion  of  these  methods  is  outside  the  scope  of 
this  monograph,  but  roughly  speaking  generalized  Fourier  series 
permit  the  most  efficient  transform  methods  to  be  developed 
followed  by  associated  Chebyshev  expansions  and  then  surface 
harmonic  expansions  but  surface  harmonic  expansions  are  best 
with  regard  to  the  pole  problem.  A variety  of  applications 
of  these  methods  to  global  atmospheric  flows  have  been  made. 

(v)  Semi-infinite  or  infinite  geometry  - Here  Chebyshev 
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expansions  are  best  if  the  domain  can  be  mapped  or  truncated 
to  a finite  domain  without  serious  error.  There  are  two  cases 
here:  additional  boundary  conditions  may  or  may  not  be 
required  at  'infinity.'  Here  again  the  formulation  of  spectral 
methods  follows  closely  the  exact  mathematics.  If  additional 
boundary  conditions,  like  radiation  or  outflow  boundary  conditions, 
must  be  imposed  on  the  truncated  domain,  then  they  should 
also  be  applied  to  the  spectral  method.  On  the  other  hand, 
if  mapping  without  additional  boundary  conditions  does  not 
introduce  a singularity  in  the  exact  equations,  no  boundary 
conditions  at  'infinity'  are  required  in  the  spectral  approximation. 
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3:  Acton  (1970),  Courant  & Hilbert  (1953),  Erdelyi  et 
al  (1953),  Fox  « Parker  (1968),  Isaacson  & Keller  (1966), 
Lanczos  (1956),  Orszag  & Israeli  (1974),  Rivlin  (1969), 

Szego  (1959),  Zygmund  (1935). 

4:  Godunov  6 Ryabenkii  (1963),  Kreiss  (1962),  Kreiss  & 

Oliger  (1973),  Laptev  (1975),  Miller  & Strang  (1965), 

Richtmyer  & Morton  (1967). 

5:  Barnett  & Storey  (1974),  Bartels  & Stewart  (1972), 

Chorin  et  al  (1977),  Lie  & Engel  (1888),  Strang  (1960), 
Richtmyer  & Morton  (1967)  . 

6:  Fornberg  (1975),  Kreiss  & Oliger  (1973),  Lanczos  (1956, 
1966),  Lyness  (1974),  Orszag  (1971c, 1972) , Wengle  & Seinfeld 
(1977) . 

7:  Price  & Varga  (1970),  Richtmyer  & Morton  (1967), 

Swartz  & Wendroff  (1969). 


9s  Gazdag  (1976),  Gottlieb  & Gustaffson  (1976),  Kwizak 
& Robert  (1971),  Mesinger  6 Arakawa  (1976),  Richtmyer 
& Morton  (1967)  , Roache  (1972)  , Widlund  (1966)  . 

10:  Cooley  & Tukey  (1965),  Cooley  (1967),  Metcalfe  (1974), 
Orszag  (1970,  1971c,  1974),  Orszag  & Israeli  (1974), 
Patterson  & Orszag  (1971). 

11:  Boris  & Book  (1976),  Browning  et  al  (1973),  Dubiner 
(1977),  Lanczos  (1966),  Orszag  (1971a),  Orszag  & Israeli 
(1974),  Orszag  & Jayne  (1974). 


12:  Haidvogel  (1977),  Roache  (1972). 

13:  Deville  & Orszag  (1977),  Orszag  (1971b, d, 1976a) . 

14:  BuZbee  et  al  (1971),  Concus  & Golub  (1973),  Metcalfe 
(1974),  Orszag  (1974),  Orszag  & Israeli  (1974),  Tang  (1977), 
Zang  & Haidvogel  (1977). 

15:  Armstrong  et  al  (1970),  Bourke  (1972),  Boyd  (1977a, b) , 
Collins  & Dennis  (1973) , Deville  & Orszag  (1977) , Eliasen 
et  al  (1970)  , Fox  & Orszag  (1973)  , Francis  (1972)  , Gazdag 
(1975),  Grosch  & Orszag  (1977) , Herbert  (1976),  Herring 

et  al  (1974),  Hoskins  (1973),  Israeli  & Orszag  (1976), 

(1”2>;  MerUees  (lo73)  , Meriiees  & Orszag  (1977) 

1976a  bl977?Phn  1971  ' ?urdock  <1977> , Orszag  (1970,  1974, 
1976a, b, 1977) , Orszag  & Israeli  (1974),  Orszag  & Pao  (1974) 

s'orszag  util)*™  ^ ' SChai"el  & «97~#  ' 
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Appendix.  Properties  of  Chebyshev  Polynomial  Expansions 

The  Chebyshev  polynomial  of  degree  n,  Tn(x),  is  defined 
by 


t (cose ) = cos  n e . 
n 


(A.l) 


Thus,  Tq  (x)  - 1,  T^x)  = x,  T2(x)  = 2x2-1,  T3(x)  = 4x3-3x( 

4 

T^ (x)  = 8x  -8x+l,  and  so  on.  Some  properties  of  Chebyshev 
polynomials  are 


| Tn ( x ) |<iv 


|T^(x) | <n2. 


(A. 2) 


~r  T (±1)  = (±l)n+p  Pn1  (n2-k2)  / (2k+l)  , 

dxP  n k=0 


(A. 3) 


Tn(x)  I*  °(n2p)  ( n-+-» ; p fixed)  , 


(A. 4) 


Tn(±l)  = (±l)n  , T2n(0)  = (-l)n,  T2n+1(0)  = 0,  (A. 5) 

T2n(0)  " T2n+i(°)  = (-1>V 

The  following  formulae  relate  the  expansion  coefficients 

a in  the  series 
n 


f(x)  * J an  Tn(x)  ( |x| <1) 


to  the  expansion  coefficients  bfi  of 


Lf(x)  = l b T (x) 
n-0  n n 


( I x I <1) 
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for  various  linear  operators  L.  We  use  the  constants 
and  dR  defined  by 

c0  * 2,  cn  * 0 (n<0),  cn  = 1 (n>0) , 

d * 1 (n>0)  , d = 0 (n<0)  . 
n — n 

Some  formulae  are: 


Lf  = f ' (x) : cRbn 


2 l Pap 

p»n+l  p 
p+n  odd 


(A. 6) 


Lf  * f"(x) : cnbn 


p»n+2 
p+n  even 


/ 2 2. 

p(p  -n  ) a 


(A. 7) 


Lf  ■ xf(x):  bn  2 ^Cn-lan-i^an+l^ 


(A. 8) 


Lf  . x2f <x)t  bn  - i 


(A. 9) 


Lf  - x4f(x),  bn  = <cn-4a„-«+(°n-Hi-2*,en-2,V2  (A. 10) 


2 2 

+ (c  ~ + 2c  ,+c  ,+c  +c  )a  + (c  ,+c  +c  ..tc  ._)a  _,-+a  ^ } 

n-2  n-1  n-1  n n n n-1  n n+1  n+2  n+2  n+4 


Lf 


f (x) -f ( 0)  . 


c b 
n n 


- P-n-1 

5 — 

= 2 l (-1)  a 
p=n+l  ^ 

p+n  odd 


(A. 11) 


-269- 


f (»)-{  (0)-f  1 (0)1  :c  b 

Lf  ~y  n n 

x 


>-n-2 


2 l (p-n) (-1)  ^a 
p=n+2  p 

p+n  even 


(A. 12) 


Lf  = 


f ' (x)-f 1 (0) 


c b - 4 \ pa 

n n p=n+2 

p-n^2  mod  4 


(A. 13) 


f ' (x)-f ’ (0)-f"(0)x  . c h = 2 l (p-n+1) pa_ 
Lf  = 2 * n n p=n+3  p 

x p-nH3  mod  4 


* 

- I (p-n-l)pa 

p=n+5  pl 


p-nEl  mod  4 


Lf  = xf ' (x)  s |c  b = na  +2  I Pap 
n n 11  p=  n+2 

p+n  even 


+ 4 l pa  ) 
p=n+  3 H 

p+n  odd 
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(A.14) 


(A. 15) 


Lf  • x2f(x):  » |iln-lla„.1*ln.ml*Vl*cn-llVl  (A.16) 


Lf  ■ xf " (x)  : c b 
n n 


2 2 


2n  (n+1)  a . + l P<P  ”n  _1) 
n+1  p-n+3 

p+n  odd 


Lf  - x f " (x) ; cnbn  = 


n(n-l)a  + 

n p=n+2 
p+n  even 


l p(p2-n2-2)a 


L 


_ f(x) 
Lf  * — j 


1-x* 


with  f(±l)-0  s cnbR  = 


-2  l (p-n)a 

p=»n+2  " 

p+n  even 


Also,  if  we  expand  f(q)(x)  as  in 


Zq  f<*>  * l VX>' 

dx  n*0 


then 


(q-1) 


n+1 
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( A . 1 7 ) 


(AM'S) 


(A. 19) 


(A. 20) 


